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Foreword 


The Human Resources Research Council was established by an Act 
of the Alberta Legislature in March 1967 ''to develop, revise and assess 
plans, materials and procedures relating to or affecting educational, 
social, economic and other aspects of human resources development, and 
to disseminate the results of and assist in the implementation of the 
findings of such developments, revisions and assessments .'' 

This monograph is intended to facilitate this end by relating 
the conceptual, methodological and administrative components of evaluative 
research to the general area of manpower policies and programs. 

Part one consists of an abbreviated discussion of the types and 
categories of evaluative research, particularly as they pertain to man- 
power studies. Included in this discussion is a summary of some of the 
major theoretical, methodological and administrative problems endemic 
to the efficient and effective evaluation of manpower programs and 
policies. Strengths and weaknesses of various manpower programs and 
activities are assessed in the light of what might be considered as an 
"evaluative ideal.'' This document is intended as a critique of the way 
evaluation of manpower policies and programs has been done to date and 
the kind of problems this has led to. It does not deal directly with the 
topic of how to do a good evaluation. For such information the reader 
should consult Suchman (1967). 

Part two is an annotated bibliography incorporating monograph- 
specific citations.) (For ithe: most part ,@it ds restnicted to materials 


dealing specifically with the evaluation of selected manpower programs 


and policies, and is by no means exhaustive. In fact, the bibliography 
does not include any citations to materials published after 1969 -- a 
rather serious limitation. Nevertheless, the bibliography is still val- 
uable and is included here. 

The views expressed in this monograph are those of the authors 


and do not necessarily represent those of; the, Counc - 
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PART ONE 


CONCEPTUAL, METHODOLOGICAL, AND 


ADMINISTRATIVE CONSIDERATIONS 


INTRODUCTION 


During the decade of the 1960's, there has been a surge of 
activity in the development of manpower policy and programs in Canada 
and the United States. For the most part, these programs have focussed 
primarily on action rather than on evaluating outcomes. 

As such, many writers -- for example, Somers and McKechnie (1968) , 
Mangum (1968 a), Hardin (1969), Evans (1969), and Bateman (1967) -- have 
commented on the difficulty of assessing whether such programs ever 
achieve their specified goals. However, as the emphasis on human resource 
development increases (placing additional strains on government and pri- 
vate agency budgets) more attention is gradually being focussed on the 
private and public rates of return on investments in this area (Dymond, 
1969:43). As a consequence, increased pressure is being placed on public 
service and community program workers to evaluate their activities. On 
surveying the literature in the field of manpower program evaluation, 
however, One must come to agree with Cain and Hollister (1969:119) when 
they suggest, "existing evaluations of social action programs ... have 
fallen short of meeting the standards possible within the disciplines 
of the social sciences''. 

Most of the evaluations carried out thus far are of relatively 
low quality. Since the application of cost-benefit studies to manpower 
program evaluation, there has been a general shift away from the more 
general, subjectivistic appraisal-type report toward the more carefully 
planned evaluative research report which assesses specific objectives 
and uses scientific methods of data collection and analysis (Suchman, 1967:19). 


While cost-benefit studies represent a significant step in the determination 
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of the economic consequences of manpower programs, they too have definite 


problems and limitations (Hardin, 1969:97-118). As Dymond (1969:46, 44-45) 


argues: 


Benefit/cost studies measure what can be measured, 
but this is all they do. They are certainly to be pre- 
ferred over speculation or political or administrative 
intuition, but they are only part of a larger context. 
If the social benefits of a program stand very high, 
they can well offset low or hegative ratios for the 
economic measures of the benefits of a program. 


Furthermore, he cautions: 


lt is important for all of us to be aware of the 
limitation of this kind of quantitative analysis. 
These analyses assume that non-economic variables 

are exogenous to the analysis and proceed on the 
assumption that such variables remain unchanged 

while the economic parameters and variables with- 

in the model change. This, of course, imposes 
constraints upon the use of these models ina 
decision-making context. There are probably feedback 
relationships between these social and economic 
variables in any situation or program. It is 
important in the process of making assumptions that 
we do not assume away an integral part of the problem, 
that we arrive at facile conclusions, or that we 
parade over-simplified quantitative measures and 
ratios in contexts in which meaningful social and 
human values are equally more important. 


In this report we shall review the current state of evaluation 
research, particularly insofar as this is reflected in the design and 
implementation of existing manpower programs. For purposes of this 
analysis manpower program evaluation is arbitrarily categorized into 
two major problem areas: 

(A) conceptual and methodological considerations, and 


(B) administrative considerations. 
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CONCEPTUAL AND METHODOLOGICAL CONSIDERATIONS 


Current evaluations of manpower programs do not permit conclusive 
judgements about the role such projects play in bringing gains to individuals, 
to governments, or to society. In large part, this inconclusiveness is 
due to the complexity of the behavior which the programs are attempting 
to change, the comparatively short existence of manpower programs and 
their lack of evaluation, the relative primitiveness of our measuring 
instruments, and open contradictions between the demands of program 
administration and objective evaluation (Evans, 1969:568-581) . 

While many writers have emphasized the need for increased 
scientific rigor in the evaluation of manpower programs, two major faults 
distinguished by Somers and McKechnie (1968:26) still characterize most 
studies: 

(1) the ''goals'' of the programs are usually so 

broad , ambigious or non-existent that it is 

impossible to ascertain the extent to which 

they might have been realized; and 

(2) the investigator fails to provide evidence 

to demonstrate that the benefits might not 

have been attained more efficiently by some 

alternative means. 
Meeting the requirements of these criteria is undoubtedly a necessary 
precursor to the development of meaningful program evaluation. In the 
meantime, we are left knowing little about whether or not public and 
private manpower programs are achieving their desired ends. Because of 
the many conceptual and methodological weaknesses endemic to present 
manpower studies, our realm of knowledge is appreciably less than our 


realm of ignorance (Somers and McKechnie, 1968:25). 


The conceptual and methodological weaknesses of present manpower 


studies are of three kinds: 


(1) ambiguity surrounding the nature and role 
of evaluation research in general; 


(2) questionable assumptions implicit in man- 
power programs; and 


(3) lack of an adequate theoretical framework 
in which to assess the viability of manpower 
programs. 
Ambiguity in the Nature and Role of Evaluation Research 
Perhaps the most basic confusion in this area concerns the nature 
of evaluation. Suchman (1967:23) has stated, ''evaluation today suffers 
from the lack of any systematic analysis of the theoretical, methodological 
and administrative principles underlying its objectives and procedures". 
To begin with, there is no clear-cut understanding of the basic requirements 
of evaluation and evaluative research. A distinction needs to be made 
between evaluation as ''the general process of judging the worthwhileness 
of some activity regardless of the method employed'', and evaluative research, 
as ''the specific use of the scientific method for the purpose of making 
an evaluation!’ (Suchman, 1967:31). This does not necessarily rule out 
other nonscientific methods of evaluation since: 
[t_]here are many evaluational questions in program 
planning, development and operation which can be 
answered without research and many, in our present 


state of knowledge, that cannot be answered even if 
the best research techniques are used (Suchman, 1967: 


SPARS 

While evaluation may be carried out using a number of different 
techniques, the evaluator must first distinguish if the particular tool 
employed is appropriate to the evaluation. If the nature of the evaluation 
dictates a rigorous scientific approach, a mere subjective appraisal will 


not suffice. Cain and Hollister (1969:133) contend that manpower programs 
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cannot realistically be evaluated with a loose research design and a 
journalistic assessment, since such an approach is unlikely to be sensitive 
to the consequent effects of such programs. 
In addition to the ambiguity surrounding the nature of 

'evaluation'', there are also varying degrees of controversy regarding 
the differing types of evaluation. For discussion purposes these have 
been categorized as follows: 

a. micro and macro evaluation, 

b. long-term and short-term evaluation, 

c. evaluation of effects, effectiveness, and efficiency; and 


d. ex post facto and a priori evaluation. 


Micro and macro evaluation. While manpower programs can be 
assessed on both a micro and macro level, most assessments have been 
made on the micro level of analysis. This was done, for the most part, 
for basically pragmatic considerations -- it is exceedingly more difficult 
to assess the impact of manpower programs at a macro level. 

Mangum (1967:7), for example, has identified six macro level 
objectives with respect to Manpower Development and Training in the 
United States. These include: 

) facilitating employment of the unemployed, 
) reducing poverty, 

) lessening inflationary pressures, 

) meeting labour shortages, 

) upgrading the labour force, and 

) revamping traditional institutions. 

Seldom have manpower evaluations endeavored to assess the 
extent to which all of these legislative goals are actually realized. 


Instead, program evaluation has been generally restricted to an extensive 


appraisal of various micro level considerations. 


of he 


This concentration on a select number of program products 
or outcomes seems to have evolved from the cost-benefit tradition. As 
Dymond (1969:50) noted, this tradition has certain positive features: 
Benefit/cost analysis imposes a very valuable 
discipline on the thought and decision-making pro- 
cesses of administrators and policy makers. It 
requires a clear articulation of objectives and a 
precise methodology for determining exactly how these 
objectives are being met. 
Cost-benefit analysis attempts to provide a method for the 


evaluation of manpower programs. This method has five essential steps: 


(1) define all the relevant benefits and costs 
for the program to be evaluated; 


(2) measure each benefit and each cost; 
(3) price each benefit and each cost; 


(4) adjust for the time dimension of these benefits 
and costs; and 


(5) apply a decision rule to the adjusted benefits and 
costs. A frequently used rule is: If the adjusted 
benefits are greater than the adjusted costs, the 
program is worthwhile; if the benefits are less than 
costs, the investments are not worthwhile (Goldfarb, 


1969:63). 
Although this procedure appears straightforward, there are major 
difficulties inherent in its application. The three most common criticisms 
of this approach are outlined by Goldfarb (1969:59-60). First, cost- 
benefit analysis is not appropriate to instances where there are significant 
nonmeasureable or nonpriceable benefits and costs. The extent to which 
manpower training increases the probability of being employed, for 
example, is not readily measurable. Such questions of measurability and 
priceability are often afforded insufficient consideration, even though 
doubts arising from these questions may completely invalidate seemingly 


meaningful results. 


ce, 


Second, the technique employed by most studies for measuring 
'benefits'' is often inadequate. This technique generally involves a 
comparison of the income and employment histories of trainees with those 
of a "control group'' of nontrainees. According to Goldfarb (1969:60): 

Many of these studies [manpower |] have failed to 

construct control groups which are acceptable in 

terms of control group methodology; furthermore, 

even an ''ideally constructed'' control group 

would not fulfill the required function and would 

lead to seriously biased results. 

Third, most general yet perhaps most important, a given program 
is one instance in a continuing series of similar programs; it is important 
to recognize the ''learning potential'' of other types of information 
which cumulating experience provides. Restricting oneself to the use 
of cost-benefit results ''contaminated'' by errors of inexperience to judge 
the potentiality of future programs seems a highly questionable procedure 
given the narrow perspective of a strictly cost-benefit approach. 

The generally accepted mode of program evaluation identifies 
the given program's objectives and assesses the degree, the efficiency, 
and the costs at which these objectives have been achieved. This structuring 
of evaluations around a few distinct considerations (generally those which 
are quantifiable) has frequently resulted in a failure to appreciate the 
fundamental role of specific program components. Evaluators have often 
been simply unable to ascertain the extent to which various program 
components actually contribute to the attainment of program objectives. 
Cain and Hollister (1969:140), for example, speculate that the success 
of programs of compensatory education might well be a function of little 
more than the ''warmth and enthusiasm'' of its teachers. All too frequently 


such inputs are discharged as insignificant, secondary considerations; 


when, in essence, they might be of paramount importance. With regard 


== 


to using cost-benefit results as the sole criteria for decision making, 
Dymond (1969:45-46) has expressed the following reservation: 
| think it is important, and | stress this point 
strongly, that | do not consider benefit/cost ratios alone 

as providing sufficient justification for choosing 

between programs or even for substantial alterations 

in existing programs. The technique is not one which 

will allow you to conclude that because there is a higher 

benefit/cost ratio in adult training as compared to a 

mobility program or an immigration program or an 

Employment Service activity, that one should put more 

resources into one program than into another or that 

one should entirely scrap programs with benefit-cost 

ratios less than unity. Except at the margins of 

resource use, programs are not substitutable for one 

another as they often serve different publics and in 

some instances, serve different functions in the 

economic process and in their social role. 

Adequate program evaluation must, in summary, not only assess 
the extent to which the program as a whole accomplishes its objectives, 
but also appraise the degree to which individual program components 
contribute to the ''success'' of the total program. These, in turn, must 
be juxtaposed against the larger macro goals of manpower programs. Unless 


this is done, evaluations and all generalizations emanating from such 


evaluations, will remain of questionable utility. 


Long-term and short-term evaluation. While manpower programs 
may be evaluated according to long-term, intermediate, or short-term 
objectives, most evaluations are restricted to a short-term frame of refer- 
ence. Evaluations of retraining, for example, generally concentrate on 
immediate income changes or employment gains as the principal criteria 
of program ''success'' (Hardin, 1969:108). 

There are numerous problems inherent in this procedure. First, 
these so called "hard'' measures of program output (which represent 


basically short-term phenomena) might well be of less significance than 
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certain ''soft'' measures such as attitudinal or motivational changes 
(which carry longer-term implications). 

Second, even if one considers changes in income and employment 
as valid indicators of program ''success'', there are still difficulties 
in making such measurements. For the most part, problems arising from 
before-after comparisons and unacceptable ''control group'' methodology 
are so extensive and sufficiently well documented as to be outside the 
realm of this document. See, for example, Hardin (1969), Cain and Hollister 
(1969), Sewell (1967), Dymond (1969), and Goldfarb (1969). 

Even should such before-after or control-experimental group 
comparisons be considered desirable, adequate follow-up data is seldom 
available. Mangum (1968 b:164), speaking of the federal manpower programs 
in the United States commented: 

For no program are there adequate valid data 

for evaluation of strengths and weaknesses and no 

program currently has a reporting system capable 

of producing such data. Data on the characteristics 

of enrollees are adequate in some but not all programs. 

Data on services provided are weak and follow-up 

data on program results are grossly inadequate and 

undependable. Ad hoe internal evaluations have been 

made of several programs, either in-house or by 

contract, but for the most part, their coverage 

is limited, their data weak and their investigations 

not probing. 

The inadequacy of before - after comparisons, and the need for valid 
follow-up studies takes on additional significance when we consider 
that the real payoff of retraining is a long-term proposition, and that 
the advantages gained by retraining might well begin to diminish after 
several years of general economic expansion (Somers, 1953:250, Somers 
and McKechnie, 1968:34). 


A third problem arising from the tendency to focus on short- 


term evaluation concerns the effect of aggregrate demand on employment 
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conditions, and consequently upon program success. If, for example, we 
attempt to isolate the gains from training, what really interests us is 

how the participants in the training schemes fared after they had completed 
training as compared with how they would have fared had they not undergone 
training. Before and after comparisons of the records of trainees may not 
satisfy this criterion simply because cyclical conditions may change from 
the pretraining to the post-training period, affecting the individual's 
economic prospects irrespective of whether he receives training or not. 
Similarly, an analysis which attempts to relate the personal characteristics 
of program participants to employment success or failure (at one or two 
points in time) suffers from a serious structural bias since it overlooks 
the fact that during ''loose'' labour market situations employers will be 

far more demanding of employee characteristics than when the labour market 
is ''tight'' (Goldfarb, 1969:74). Evaluations which do not transcend the 


immediate are, for these reasons, likely to be questionable. 


Evaluation of effects, effectiveness, and efficiency. Suchman 
(1967:61-71) has outlined five categories of criteria according to which 
the success or failure of a program may be evaluated. These include: 


(1) Effort -- This represents an assessment of input or 
energy, regardless of output. 


(2) Performance -- This represents an assessment of the 
results of effort rather than effort itself. 


(3) Adequacy of performance -- This criterion refers to the 
degree to which effective performance is 
adequate to the total amount of need. 


(4) Efficiency -- This represents an assessment of alternative 
paths or methods in terms of costs. 


(5) Process -- The emphasis is on learning how and why a 
program works or does not work. 
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Virtually all of the earlier attempts at manpower evaluation 
utilize effort as the principal criterion of program success. These 
attempts have been essentially journalistic assessments, relying extensively 
on information already contained in management records. While this kind 
of evaluation has the advantage of being relatively easy to perform, its 
findings contribute very little to our understanding of manpower programs 
in general (Suchman, 1967:61). 

Performance and adequacy of performance criteria have been 
widely employed in the analysis of cost-benefit studies, as well as in 
the assessment of United states Manpower Development and Training (M.D.T.) 
and Area Redevelopment Administration (A.R.A.) Scape all While these 
criteria have their particular strengths and weaknesses, there is a general 
tendency to dispel both in a superficial manner. The following comment 
on adequacy of performance is typical of the brevity with which these 
criteria are usually dealt: 

The gains to society resulting from the retraining 

program are not so readily documented or evaluated. 

As has been noted, a reduction in unemployment for 

particular trainees does not necessarily imply that 

retraining programs have reduced total employment. 

They may merely have contributed to a change of faces 


in the unemployment compensation and welfare queues 
(Somers, 1956:291). 


See, for example: Blume (1968), Borus (1968, 1970), Cain 
(1967), Cain and Somers (1967), Cain and Stromsdorfer (1968), Chapman 
(1966), Cornelson (1964), Hardin and Borus (1966), Kirby and Castagna 
(1969), Main (1968), Mangum (1967), Page (1964), Solie (1968), Somers 
(1965 b), Somers and Stromsdorfer (1965), Stromsdorfer (1968), and Walter 
and Magnusson (1967). 
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Similarly, the fundamental inability to delineate the role of individual 
program components (as outlined in the section on micro and macro evaluation), 
has resulted in assessments of performance which are, at best, superficial. 
Suffice it to say, neither performance nor adequacy of performance criteria 
are sufficiently well understood to provide results which are both valid 
and meaningful. As a consequence, many manpower programs are retained, 
modified and done away with -- without an appropriate understanding of 
their ''success'' (whether defined as the results of the effort, or the extent 
to which effective performance is commensurate with need). 

The preceding comments also apply to efficiency evaluations. 
Manpower programs are rarely analyzed for their relative success. This 
inevitably leads to a narrowness of perspective which selectively screens 
out other potentially relevant solutions to manpower problems. In discussing 
the worthwhileness of youth employment projects administered under the 
United States Economic Opportunity Act, for example, Miller and Rein 
(1965:289) have expressed the following concern: 

Would increasing the income of a poor family do as 

much to increase the children's interest in further 

advance as would specific training programs? We are 

not confident that the results of a study of these 

alternative ways of spending poverty funds would 

Support a training program, separate from improving 

the economic conditions of families and neighborhoods 

Economic and social change may have to precede the 

call for individual responsibility and adjustments. 
Goldfarb (1969:61) has also commented on this point. While there are 
a number of justifiable or valid rationales (i.e. market imperfections 
and the existence of poverty) for government sponsored retraining, he 
cautions: 

Even if these arguments are valid and applicable, 


they do not necessarily justify training programs. 
Programs to disseminate information, loan programs 


athe 


to counteract imperfect capital markets, or other 
government-supported devices may prove to be far 

more efficient at attaining the objectives desired. 
The choice of programs requires a study of the rela- 
tive efficiency of various possible manpower programs. 


This tendency to evaluate individual manpower programs as isolates has 
resulted in a fragmentation of knowledge which obscures existing alterna- 
tives and defies comprehensive policy formulation. 

Although process evaluation is prerequisite to a more 
adequate understanding of manpower programs, it has been employed only 
infrequently and incompletely. For the most part, the analysis of process 
may be considered to have four elements: (1) the attributes of the program 
itself; (2) the population exposed to the program; (3) the situation or 
context within which the program takes place; and (4) the different kinds 
of effects produced by the program (Suchman, 1967:67). Only when these 
four are systematically explored, is it possible to comment intelligibly 
on how and why a program succeeds or fails to succeed. Goldfarb (1969:60), 
in warning against the increasing reliance on cost-benefit measures (which 
examine performance and adequacy of performance criteria and result in 
a narrowness of perspective), stresses the importance of process evaluation: 

Since a learning process must take place before pro- 

grams begin to perform their tasks efficiently, and 

since speeding this learning process will result in 

better programs sooner, it may be highly advisable to 

re-define the evaluator's task to include aiding the 

learning process. Since a meaningful evaluation must 

investigate how closely the program being studied approx- 

imates some level of ''reasonable efficiency'', the eval- 

uator has the information at hand to suggest ways in 

which program administration can be improved. In fact, 

useful suggestions from the analyst about how to better 

promote the training program's social aims would seem 

to make a far larger contribution to social welfare than 


the finding that Program A had a benefit-cost ratio of 
1: /eOgeoTides. 
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EX post facto and a priori evaluation. The almost exclusive 


reliance On an ex post facto approach to program evaluation has severely 
restricted what may be learned from evaluating manpower programs. Cain 
and Hollister (1969:148) summarize this deficiency as follows: 


A program is built around a single concept; eventually 

it is realized that it does not work and a new program 

and concept is tried. An intentionally experimental 
approach would allow us to learn faster by trying alter- 
native concepts simultaneously and would make it more likely 
that we could determine not only that a particular concept 
failed, but also why it failed. It takes restraint to mount 
a program with a built-in experimental design and wait for 
it to mature before deciding on a single program concept. 


This phenomenon takes on even greater significance when one considers 
certain other aspects of the total learning experience. Somers (1965 b:278) 
provides one such insight when he writes: 


Given the fact that almost all MDTA and ARA trainees 
are unemployed before their enrollment in a training 
course, a /0 to 75 percent after-training employment 
ratio seems to be a very respectable showing. But 
before this result can be chalked up as a solid gain 
attributable to retraining, some nagging questions 
must be faced. Is the government training and placing 
primarily those who would be capable of finding 
employment without the benefit of retraining? This is 
an especially pertinent question, since in the selection 
of persons for training there has been a tendency to 
choose those who are relatively younger and better 
educated than the unemployed as a whole. On the other 
hand, there are more long-term unemployed among those 
selected for training than among the total unemployed 
work force. 


Hopefully, as evaluative research becomes more accepted and the methodology 
more refined, provision for continuing evaluation will become an integral 
part of overall research designs. The evaluator may then become involved 
earlier and collect data more extensively for the duration of the program. 
In this way, both the products of the program and the process by which 


they came to be attained can be scrutinized. This in turn will provide 


Shae 


a more comprehensive understanding of the functioning and adequacy of 


present manpower programs and practices. 
The Basic Assumptions of Manpower Programs 


The second problem area to be discussed under the category 
of conceptual and methodological considerations concerns the basic 
assumptions which are contained (either implicitly or explicitly) in 
manpower programs and policies. Increased reliance on manpower programs 
as a means for ameliorating such national problems as poverty and unemploy- 
ment suggests confidence in the principle of manpower development. For 
a number of reasons, this confidence may be unfounded. First, we simply 
do not have enough valid information on the impact and functioning of 
manpower programs to arrive at definitive conclusions regarding their 
success or failure. This predicament is epitomized by Somers (1965 c: 


206-207) when he writes: 


The programmes established under recent retraining 
legislation in Western Europe, the United States 
and Canada must be considered as substantial invest- 
ments by the government in human capital. When the 
subject is viewed in this light one naturally asks: 
"What is the rate of return on this investment?", ''ls 
it a wise investment?'', ''What are the gains and costs 
of the retraining investments?'' Although some research 
has been conducted along these lines, there are still 
grave methodological problems in reaching valid con- 
clusions ... Almost all of the statements, studies and 
analyses dealing with this question conclude that 
training and retraining is a sound investment 
for the trainee and for society. But in spite 
of these claims for the high rates of personal 
and societal return to the retraining investment 
-- and the author's own predilection to accept 
such claims -- it must be noted, again, that there 
is little definite proof -- based on a careful 
research design -- of such a rate of return. 
Even if trainees do much better in the job mar- 
ket than control groups of non-trainees -- as they seem 
to do in almost all our studies -- one cannot be certain 
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that the job record is attributable to retraining. 

lt may result from the trainees' greater youth, 

higher education, etc; or from the Employment Service's 

greater efforts as job placement of trainees vis-a-vis 

non-trainee job applicants. One can accept the economic 

values of retraining on faith; but it is no simple 

matter to lend quantitative precision to one's beliefs. 

This represents a major challenge to the research 

investigator. 

Second, since the raison d'étre for manpower programs has seldom 
been challenged, little consideration is given opportunity cost assessments 
whereby the viability of one program is examined vis-a-vis some other 
alternative. Accordingly, little effort is devoted to testing the complex 
set of assumptions implicit in manpower development. The underlying rationale 
for most programs of retraining, for example, is provided by the expectation 
that despite the relatively high level of unemployment prevalent in recent 
years, there exist a good number of unfilled employment opportunities for 
those who can acquire the necessary preparation. The analysis of numerous 
government sponsored retraining programs (in which only minor reductions 
in unemployment were achieved) provides accumulating evidence to contradict 
this generally accepted premise. As Wiessman (1969) has suggested, the 
best kind of training in the world will not suffice if the economy does not 
produce enough jobs, and schools do not provide skills necessary to master 
such jobs. The problem of unemployment cannot be solved apart from the 
general problem of employment in the economy, for no amount of training 
will suffice if there is no need for workers. 

A third reason for questioning the confidence in manpower activities 
is provided by Somers (1965 a:228,233) in his discussion of United States 
A.R.A. and M.D.T.A. sponsored training programs: 

lt has been pointed out that in the selection of 


trainees the government-sponsored training programs 
have by-passed most of the hard-core unemployed, 
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that many workers have been trained for jobs which 
do not exist, and that the workers who benefit most 
from the training programs are the very ones who 
could find employment without government aid. 


Furthermore, he continues: 
Notwithstanding the relatively high placement 

ratios reported under both programs, it is pertinent 

to inquire to what extent the present government 

retraining programs have created more employment for 

the unemployed and how secure the employment position 

of trainees is in the months following their training. 

These are obviously basic questions. 
As these comments suggest, there are undisclosed and unaccounted contingencies 
which may well function to misrepresent seriously the viability of existing 
manpower programs. For example, many evaluators feel that manpower programs 
continue to train workers for occupations which will soon be rendered obsolete 
by technological advance (Somers, 1965 a:233; Somers, 1965 c:201). There 
is sufficient evidence, particularly in the case of the earlier attempts 
at manpower development, to indicate that the hard-core unemployed are 
omitted from programs of training and that trainees represent, in most 
instances, the ''cream'' of the human material available for training (Miller 
Sted bo 1970). There is also much concern that the composition of trainees 
in many manpower programs is grossly unrepresentative of the total pop- 
ulation of the unemployed (a consequence of employing selection criteria which 
excludes certain categories of the unemployed such as the older, less educated, 
or long-term unemployed worker) (Somers, 1956 a:235-236) . 

A very elemental consideration (with gross implications for all 
public and private efforts aimed at assisting the ''poor'') concerns the 
assumption that work is a basic good and as such is appealing to all persons 


who are not employed. Even though very little is actually known about the 


role of work in the real life experience of those who are regularly employed 
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(Goodwin, 1969: 508-519), it is assumed that employment is meaningful and 

this assumption, in turn, is generalized (perhaps quite erroneously) to 

the unemployed. Certainly, the role of work in the lives of the poor is 

not sufficiently well understood to make such conclusions justifiable. 

Given the rather narrow range of employment opportunities traditionally 
afforded the poor, and the rather formidable mosaic of contingencies restrict- 
ing upward mobility, it seems unlikely that a good many of the poor perceive 
work as either desirable or meaningful. Furthermore, there are many instances 
where employment is not a viable solution to poverty (eg., a large family 
headed by an uneducated mother), and where some alternative system of income 
maintenance is undoubtedly more appropriate (Levine, 1968). 

Perhaps a final feature of manpower policies and programs involves 
the pervasive ambiguity which characterizes the motivation for such efforts. 
That is, because of differing interpretations of the etiology of manpower 
problems, it is possible to have several very divergent attempts at a solution 
to the same general problem. One view of teenage unemployment in the United 
States, for example, maintains that it is caused by apathy, incompatability 
and poor employment attendance. The solution, if such be assumed, is a 
work-experience program (such as the Neighborhood Youth Corps) with some 
minimal training component. Another way of perceiving the same problem 
is to define it as a lack of marketable skills; the solution would then 
require an occupational training program similar to the Job Corps. Until 
we can decide whether manpower development is designed to remedy inadequate 
work skills, unacceptable work habits, lack of appropriate attitudinal or 
motivational constructs, or deficient employment information (to mention 
a few), we will be unable realistically to assess the extent to which any 


given program is successful in emulating its objectives. Project-by-project 
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evaluations are bound to turn up "success! somewhere, but unless such success 
can be meaningfully related to the efficient and effective dischargement of 
the program's objectives, the ensuing results will be both spurious and 
inconsequential. 

By repeatedly emphasizing positive project evaluations which are 
restricted to the attainment of low level objectives and a narrow range of the 
population in need, without sufficient regard for the assumptions implicit 
in such undertakings, we are invariably perpetrating a myth--for such 
evaluations imply that our efforts are in the right direction. Accordingly, 
it must always be remembered that: 

[tlo a great extent, the ultimate worth of evaluation for public 

service programs will depend upon research proof of the validity 

of the assumptions involved in the establishment of key objectives 

(Suchman, 1967:57). 


Any attempt at evaluation which does not include at least rudimentary pro- 


vision for such contingencies will prove less than adequate. 


Lack of a Theoretical Framework 

A third problem of analysis concerns the lack of a theoretical 
framework in which the assumptions and data concerning manpower programs 
might be tested. These deficiencies are discussed under two different 
headings: 

Lack of a precise statistical model. Cain and Hollister 
(1969:132) adequately sum up the role of theory in manpower programs 
evaluation when they suggest that the function of a theoreticl structure 
is to evolve a statistical model for testing. As they see it: 

...the theory [which includes the body of substantive 

findings from previous studies] can tell us what 

independent variables may be included as statistical 


controls for the purpose of reducing the unexplained 
or residual variation in the dependent variable. 
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Unfortunately, a good many manpower programs are designed with insufficient 
attention to the theoretical framework with the result that subsequent 
evaluations invariably disclose the ''presence of variables, not included 
in the statistical model, which are correlated with both the dependent 
variable and the treatment variable'' (Cain and Hollister, 1969:134). If, 
for example, trainees do fare better than non-trainees, we frequently cannot 
explain why. As Somers (1965 ¢:205) puts it: 

[t]he big unanswered question is whether these 

workers -- the cream of the unemployed from the 

standpoint of age and education -- would have 

obtained jobs just as readily if they had been 

bypassed for training but were given the same 

treatment as were the trainees in careful testing 

selection and placement efforts by the employment 

service. 
Perhaps the willingness of employers to hire trainees has little to do 
with the trainees' newly acquired skills. Perhaps to the employer the trainee 
represents a better employment risk -- having demonstrated a certain degree 
of ambition and drive by successfully completing a course of training. 
Perhaps such a situation represents nothing more than a policy of preferential 


hiring in which employers have agreed in advance to accept trainees before 


non-trainees (Decore et al., 1969). 


Lack of substantive theory. As noted in the beginning of the 
last section, a crucial phase of theory development involves the capacity 
to utilize the accumulated substantive findings of previous studies in the 
design of later efforts. For a number of reasons, this is not always 
easy. In many instances, the substantive findings are so inconclusive and 
suspect as to defy their integration into subsequent theoretical structures. 


Williams (1969:454) provides insight into this development when he writes: 
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In the past, much of the evaluation effort has 
focused on local project measurement. Yet these 
studies seem unlikely to be used to program (multiple 
project) planning activity. This point does not 
suggest that a local project's performance is more 
difficult to measure than a total program outcome. 
What is difficult is assessing the ''why'' of that 
individual performance, and this deficiency blocks 
generalization to other projects. A claim that a 
local project model should be replicated widely 
must be based on evaluation data showing explicitly 
that success did not derive from atypical quality 
factors (for example, a charismatic teacher) or 
from exogenous variables (for example, the level 

of local economic activity). 


In other cases, findings are often incorrectly generalized from one 
program to another. There has been, for example, a tendency to general - 
ize the results of various benefit-cost studies of training schemes 
(administered under the Manpower Development and Training Act of 1962, 
the Area Redevelopment Act of 1961, or earlier State equivalents of such 
United States federal legislation) to programs subsequently created under 
the War on Poverty. Such programs differ appreciably -- according to the 
trainees involved, the objectives of the program, and the environmentin 
which the training takes place. When the programs in question have radically 
different target populations, findings cannot always be meaningfully 
generalized. 

A good portion of the preceding problem can be categorized 
under what Cain and Hollister (1969:128-129) refer to as the ''replicability 
cebtantion'’: 

A source of friction between administrators of 

programs and those doing evaluation research (usually 

academicians) is the failure to agree upon the level 

of decision-making for which the results of the eval- 

uation are to be used. This failure, which is all the 

more serious because the issue is often not explicitly 

addressed, leads to disputes regarding two related 

issues -- the scope of the evaluation study and the 

selection of variables to be studied. To deal with 


these disputes, we suggest applying the "'replicability 
criterion''. We apply this name to the criterion be- 


ie = 


cause of the large number of cases in which evaluations 

of concepts have been made on the basis of projects 

which are not likely to be replicable on a large scale 

or which focus on characteristics of the project which 

are not within the ability of decision-makers to control. 

This criterion implies that unless there is tangible evidence 
that the ''success'' of a program can be broadly replicated and that admin- 
istrative controls are sufficiently strong to ensure replication, then 
individual program success is irrelevant and devoid of substantive meaning. 
Stated somewhat differently, this criterion affirms that resources should be 
allocated in the light of evidence that manpower programs are not only 
"successful'' on a priori grounds or in particular small scale contexts 
but are, in fact, ''successful'' in large scale implementation. Overlooking 
this consideration has been instrumental in limiting the degree of applica- 


bility of substantive findings and ultimately in keeping manpower theory 


at a low level of development. 


ADMINISTRATIVE CONSIDERATIONS 


The preceding discussion has attempted to outline some of the 
many conceptual and methodological weaknesses which have characterized 
manpower evaluations to date. Since many of the aforementioned limitations 
are related, at least in part, to certain administrative consideration, 

a discussion, of theseeis .al50 .in, onder, 

Research at the best of times involves a compromise between the 
ideal and the resources available for distribution. This is especially 
true of evaluative research, where funds may be limited and hypotheses 
very often related to vested interests. The resulting compromise 
between practicality and scientific rigor can seriously jeopardize both 


the quality of the research and the validity of the findings (Suchman, 1967:151). 
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Evaluation research, by its nature, places a strain on the work- 
ing relationship between the evaluator and the administrator and staff 
of the program being evaluated. Since the primary function of the evaluator 
is to determine whether or not a program is attaining its desired objectives, 
there is very often conflict and open resistance to his efforts. Few 
individuals with strongly vested interests in the program they are involved 
in can be expected to welcome an objective evaluation -- particularly when 
they may have been ''socialized'' into taking the worthwhileness of their 
activity for granted. Because of this inherent friction, it is sometimes 
difficult to get administrators to agree to anything more than a token 
evaluation in which the stronger points of the program are emphasized, 
while the weaker ones are downplayed or never mentioned at all. 

Similarly, the problems emanating from the ex post facto 
evaluation of manpower programs can generally be traced to the discrepancy 
between the needs of the evaluator and the needs of the administrator. 
Because of the common practice of involving the evaluator after the program 
has been designed and operationalized, information (considered imperative 
from the point of view of the evaluator) might not be available. The demands 
of program administration and objective evaluation are sometimes mutual ly 
exclusive (Evans, 1969:568-581). 

Administrators, especially at local levels, 

tend to place a low priority on data collection 

and analysis, and the result is that systems op- 

erators are seldom able to deliver on schedule the 

range of data which they originally promise. We 

have to recognize, also, that project operators 

sometimes have incentives to provide biased or 

simple manufactured data. Finally, internal data 

systems are notoriously inflexible, since the 

systems are usually designed with a limited set of 

users in mind. The result is that the analyst finds 

it impossible to obtain disaggregations of these 

data or reaggregations by different sets of class- 


ifications. The importance of conserving micro- 
data has still not been generally appreciated 


(Cain and Hollister, 1969:145). 
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The practice of involving the evaluator and constructing the 
evaluation component after the program is underway or terminated has 
certain other unequivocal implications. There may be serious problems and 
deficiencies in sampling (Cain and Hollister, 1969:134,127). If an evaluator 
is to arrive at valid conclusions concerning the value of retraining in 
placement, he must compare the post-training experience of the trainees 
with the experience of control groups who were similar to the trainees in 
characteristics and unemployment records when the training course began, 
but who have not undergone the retraining. Sometimes comparable experimental 
and control groups do not even exist. This is frequently the case when 
procedures of randomization (the random assignment of individuals to exper- 
imental and control groups) have not been employed. When this situation 
exists it is exceedingly difficult for the evaluator to determine whether 
observed differences between groups is a consequence of the program or 
merely a result of the incompatability between the two groups. 

Even when randomization has been effectuated, there is generally 
insufficient information concerning the population to ascertain whether 
the samples are representative and, consequently, the extent to which the 
evaluative results are generalizable. This lack of information regarding 
possible target populations is one of the prime reasons why the macro 
effects of programs are seldom assessed. 

The influence of administrative considerations on manpower 
program evaluation is extensive: 

[aldministrative forces are felt at the very begin- 

ning of the project in terms of the statement of 

objectives and the specification of activities to be 

evaluated. They continue to be felt in the laying 

out of the research design, in selecting the sample, 


in the collection of the data, and in the analysis 
and interpretation of the results (Suchman, 1967:161). 
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At times, the administrative component can present a rather formidable 
barrier to the systematic and comprehensive evaluation of manpower programs: 


There have been numerous cases in which months, and 
even years, have been taken up in arguments over what 
the program objectives ''really are'' or how multiple 
objectives are to be ''weighted'' to add up to some 
overall goal measure. In the meantime, programs 

have stumbled on with no evaluation or new programs 
have been forestalled because no "'a priori'' evalua- 
tion was undertaken to assess the feasibility of the 
program. Wily bureaucrats have been able to prevent 
evaluation of their programs for many months by 
refusing to ''sign off'' on a defined set of objectives 
(Cain and Hollister, 1969:124). 


A further, though closely related, concern involves the general 
acceptability of evaluation results. Because independent and objective 
evaluation frequently discloses information which challenges many of our 
traditional assumptions and beliefs, it is usually met with much skepticism 
and resistance. This often leads to the claim that evaluation research 
is not really ''scientific''; and, as such, cannot be relied upon to produce 
valid and reliable results: 


lt often seems that the scholars conspire with the 
legislators to beat down any attempt to bring to 

bear more orderly evidence about the effectiveness 

of alternative programs. It is not at all difficult 
to find experts who will testify that virtually any 
evaluation study is not adequately ''scientific'' to 
provide a sound basis for making program decisions. 
There is a reasonable and appropriate fear on the 
part of academics that sophisticated techniques of 
analysis will be used as deceptive wrapping around 

an essentially political kernel to mislead administra- 
tors or the public. This fear, however, often leads 
to the setting of standards of ''proof!' which cannot, 
at present, given the state of the art of social 
sciences, or perhaps never, given the inherent nature 
of social action programs, be satisfied. The result 
generally is that the evaluation is discredited, the 
information it provides ignored, and the decision- 
maker and legislator can resume the exercise of their 
visceral talents (Cain and Hollister, 1969:149). 


Sa 


The fact remains, however, that the use of systematic evaluation repre- 
sents a serious attempt to elevate the standards of what is admissible 
evidence for informed decision making. While problems of data, organization, 
and methods make conclusive judgments extremely difficult, this should not 

in itself be construed as a negation of the value of the further efforts 
towards attempts at evaluating manpower policies and programs: 


Higher standards of evaluation will lessen the role 

of ''hearsay'' testimony in the decision process, but 

they are not meant to provide a hard and fast decision 
rule in and of themselves. The public decision-making 
process is still a long way from the point at which the 
evidence from a hard evaluation is the primary or even 
the significant factor in the totality of factors which 
determine major decisions about programs. Therefore, 
the fear of many academics that poorly understood evalu- 
ations will exercise an inordinate influence on public 
decisions is, to say the least, extremely premature. 

But if standards for the acceptance of evaluation re- 
sults are viewed in terms of the "rules of evidence'' 
analogy, we can begin to move toward the judicious mix 
of rigor and pragmatism that is so badly needed in eval- 
uation analysis (Cain and Hollister, 1969:150). 
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PART TWO 


AN ANNOTATED BIBLIOGRAPHY OF 
SELECTED EVALUATION STUDIES 


OF MANPOWER PROGRAMS 
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This article discusses a recent evaluation of the Rough Rock 
Demonstration School (RRDS), a revolutionary attempt by Navajos 
to formulate an educational system based on their own tradition. 


Three problem areas in cross-cultural evaluation are identified 
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Pp. 140 - 154 in Thomas A. Goldman (ed.), Cost Effectiveness 
Analysis. New York: Frederick A. Praeger. 
A discussion of the use of cost-benefit analysis in the evaluation 
of anti-poverty programs. 


The article considers the training investment in the light of 

certain fundamental questions such as: (1) are training expenditures 
justified in terms of their returns relative to the return from 

an equal amount of expenditure on an alternative program? (2) how 
should resources be allocated among alternative training programs? 
and (3) for any given training program, what factors have the 
greatest influence on costs and returns? 


Blume, Paul R. 
1968 An Evaluation of Institutional Vocational Training Received by 
American Indians Through the Muskogee, Oklahoma Area Office 
of the Bureau of Indian Affairs. Stillwater, Oklahoma: 
Oklahoma State University, May, 1968. 


An assessment of a training program for 672 Indians operated by the 
Muskogee Area office during the period 1958 to December 31, 1966. The 
findings indicated that completion of training led to an increase in 
both earnings and employment. 


Borus, Michael E. 


1964 "A benefit-cost analysis of the economic effectiveness of 
retraining the unemployed.'' Yale Economic Essays 4 (Fall): 
371-429. 


A general discussion of the relative merits of governmental and 
private retraining programs. The author anticipates a positive 
benefit-cost relationship for trainees relative to a comparable 
control group. 
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A report evaluating how successful early retraining courses 
in Connecticut have been in achieving the congressional 
objectives of: (1) increasing the nation's output; (2) red- 
ucing the aggregate level of unemployment; (3) reducing the 
costs of unemployment and public assistance; and (4) reducing 
the unemployment of specific groups of the unemployed. More 
specifically, the report attempts to answer the following 
questions: (1) Do the employment records of the retrained 
workers evidence significant improvements as a result of the 
retraining program? (2) What have been the effects of 
retraining on aggregate output, unemployment and government 
unemployment payments? and (3) What groups in the labor 
force have been retrained, and are these the workers Congress 
desired to retrain? 


On the basis of the Connecticut experience, it appears that 
retraining is achieving all of its objectives except that 
of attracting specific groups. 


1968 'Time trends in the benefits from retraining in 
Connecticut.'' Pp. 36 - 46 in Gerald G. Somers (ed.), 
The Development and Use of Manpower. Washington: Industrial 


Relations Research Association. 


By examining gains in earnings resulting from vocational 
retraining, the author attempts to identify trends in 
average retraining benefits (over a five year period) and 
to determine whether such trends differ significantly 
depending on variable characteristics of the trainees. 


Borus, Michael E., et al. 
1970 "A benefit-cost analysis of the neighborhood youth corps: 
the out-of-school program in Indiana.'' Journal of Human 
Resources 5 (Spring): 139 - 157. 


Using two types of societal benefits -- the increase in the 
level of aggregate output and the increase in employability 
(ability to earn) of NYC participants -- the program dem- 
onstrates that: (1) the earnings increase as a result of NYC 
participation is substantial for males and high benefit- 

cost ratios are obtained by society; (2) female participants 
increase their post-program earnings very little and their 
benefit-cost ratios for society are almost always below one; 
(3) high school dropouts benefit more from the program than 
do graduates from high schools (therefore, the program should 
continue to be oriented towards dropouts primarily); (4) the 
longer the participants remain in the NYC, the greater are the 
increases in post-program earnings. 
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Beunaventura, Angeles. 
1967 Follow-up Study of MTDA E & D. Project Conducted by the 
Michigan Catholic Conference, Lansing. Washington D.C.: 
Bureau of Social Science Research, Inc., March, 1967. 


A report on one of eight MDTA Experimental and Demonstration 
projects sponsored by the Michigan Catholic Conference reveals 
that people with difficult employment problems are trainable. 
The report also attempts to determine whether trainability 
leads to employability, and to identify the components of 
training programs and community life which may contribute to 
training and employment outcomes. 


Bureau of Social Science Research, Inc. 
1968 Final Report on the Evaluation of Eight E & D Manpower 
‘Training Centers. Washington D.C.: Bureau of Social 
Science Research, INc., June, 1968. 


An appraisal of eight Experimental and Demonstration 
projects in which southern Negro colleges were used as 
training centers for the hardcore unemployed. The report 
concludes that training alone will not benefit the 
individual if it is not combined with corresponding job 
development and placement. 


Burke, Ronald J. 
1969 "A plea for systematic evaluation of training.'' Training 
and Development Journal 23 (August) 24 - 26. 


Discusses the impact of 33 hours of human relations 
training on the managerial attitudes and beliefs of 57 
undergraduate business students enrolled in a management 
course at a large midwestern state university. 


Cain, Glen G. 
1967 Benefit/Cost Estimates for Job Corps. Madison, Wisconsin: 
Institute for Research on Poverty, University of Wisconsin, 
September, 1967. 


A sophisticated cost/benefit analysis which attempts to 
answer the question, ''Does the investment in the Job 
Corps pass the economic test of efficiency?" Using two 
potential sources of benefits to enrollees -- 
educational gains and increased earning power -- the 
program appears to have a favorable tenefit/cost ratio. 


Cain Glen G. and Robinson G. Hollister. 
1969 "Evaluation manpower programs for the disadvantaged.'' 
Pp. 119 - 151 in Gerald G. Somers (ed.), Cost-Benefit Analysis 
of Manpower Policies: the Proceedings of a North American 
Conference. Kingston Ontario: Industrial Relations Centre, 
Queen's University. 


res Wl li 


A rather detailed discussion of the methodology of 
evaluation insofar as it applies to anti-poverty 
programs. Specifically, the theme the authors develop 

is that evaluations should be designed to provide the 
basic ingredients of an experimental situation: a 

model acceptable for statistical testing, wide range in 
the values of the variables representing the program 
inputs, and the judicious use of control groups. Several 
backgrounds are incorporated into the paper - the tradition 
of benefit-cost analyses from economics, the approach 

of quasi-experimental research from other social 
sciences; and the perspective of initiating and using 
evaluation studies from a governmental agency. 


1969 The Methodology of Evaluating Social Action Programs. 
Madison, Wisconsin: Institute for Research on Poverty. 
University of Wisconsin. 


A discussion of evaluation methodology, particularly as 
it applies to anti-poverty programs. The elements 
required to design an evaluation include specifying the 
program objectives, using control goups, agreeing on 
the level of decision-making for which the evaluation 
results are to be used, providing a statistical model 
for testing, applying economic theories, and timing the 
evaluation so that sufficient information is available. 


Cain, Glen G. and Gerald G. Somers. 
1967 "Retraining the disadvantaged worker.'' Pp. 27 - 44 
in Cathleen Quirk and Carol Sheehan (eds.), Research in 
Vocational and Technical Education. Madison, Wisconsin: 
Center for Studies in Vocational and Technical Education, 
University of Wisconsin. 


A comment on the techniques for evaluating training 

programs for hard-core disadvantaged groups as well as 

the application of these techniques to two specific 

case studies. The article indicates that welfare 

trainees were more successful in leaving the relief rolls 
than were a control group of non-trainee welfare recipients. 


Cain, Glen G. and E:W. Stromsdorfer. 
1968 "An economic evaluation of government retraining of the 
unemployed in West Virginia.'' Pp. 229 - 335 in Gerald 
G. Somers (ed.), Retraining the Unemployed. Madison, 
Wisconsin: University of Wisconsin Press. 
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An economic evaluation of a program, sponsored by 

the federal and state governments, to retrain under- 
employed and unemployed workers in West Virginia. 
Information on the labor market experience (employment 
and earnings) of a group of workers who received 
training is compared with a control group who did not 
receive retraining over the same period. Two broad 
generalizations, both with important qualifications, 
summarize the findings: 


(1) There are sizable differences in the measured 

benefits of the training program among the several 
socio-demographic subgroups - the earnings of men 

appear to have increased considerably more than the 

earnings of women, the earnings of younger men increased more 
than older men, and the lower educated groups benefited 

more than the higher-educated groups. However, smal] 

sample sizes and attendant problems of statistical 
reliablility make such comparisons extremely tentative. 


(2) Despite the heterogeneity among groups, the overal| 
result is a considerable excess of benefits over costs, 
implying that the program paid substantial dividends. 
The reservation here is that the basis for measuring 
how the program paid off contains biases which tend to 
overstate the benefits. 


Capron, William M. 
1967 'Cost-effectiveness analysis for government 
domestic programs.'' Pp. 131 - 140 in Thomas 
A. Goldman (ed.), Cost Effectiveness Analysis. 
New York: Frederick A. Praeger. 


A discussion of the use of cost=benefit or 
cost-effectiveness criteria for program analysis. 


Chamber of Commerce of the United States. 
1966 Youth and the War on Poverty: An Evaluation of the Job 
Corps, Neighborhood Youth Corps, and Project Head Start. 
U.S.: Chamber of Commerce. 


A research report designed as an aid to understanding and 
evaluating government programs of interest to the busin- 
ess community. 


Chapman, Jane R. 
1966 Oak Glen - A Training Camp for Unemployed Youth. Auto- 


mation Research Monograph N.S. United States Department 
of Labor: Manpower Administration. 
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A description and evaluation of the Oak Glen Youth Camp 
near Riverside, California where training was provided 
for unemployed youth 16 to 21 years of age. On the basis 
of data collected on 479 young men, the major findings 
of this pilot study were: (1) 70 percent of all 
graduates were employed, as compared to 55 percent of 


the terminees; (2) the more education possessed by the 
trainee the better his chances for completion and sub- 
sequent employment; (3) IQ or general aptitude was not 


a predictor of success; and (4) terminations increased 
during the latter stages of the program with disinterest 
and homesickness cited as the most frequent reasons for 
leaving. 


Chesler, Herbert A. 

1968 "The retraining decision in Massachusetts: theory and 
practice.'' Pp. 149 - 169 in Gerald G. Somers (ed.), 
Retraining the Unemployed. Madison, Wisconsin: 
University of Wisconsin Press. 


An exploratory study of the decisions of workers to 
accept or reject opportunities to participate in state 
retraining programs established under the aegis of the 
Area Redevelopment Act of 1961. The author examines 

the relationship that exists between an individual's 
decision for or against training and his selected personal 
characteristics and work history experiences. Variables 
thought to play an active role in the making of such 
decisions were age, education, marital status, attitude 
toward governmental programs and subsidies, employment 
status at the moment of decision,and employment and 
unemployment experiences immediately prior to the receipt 
of the training offer. 


Cloward, Richard A. 


1965 "Our illusions about training.'' American Child 47 (January): 
6 - 10. 


A discussion of the validity of emphasizing motivational 
training in youth employment programs instead of stress- 

ing educational and occupational upgrading. The author 
maintains that disadvantaged youth: (1) realize that few 
jobs await them upon completion of training and are hesitant 
to enter employment programs; and (2) have neither the 
formal education nor the skills to become competent in any 
specific occupation. 
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Cohen, Malcolm S. 
1969 "The direct effects of federal manpower programs in 
reducing unemployment.'' Journal of Human Resources 


4 (Fall): 491 = 507. 


Using four manpower programs - Neighborhood Youth Corps, 
On-the-Job Training under the Manpower Development and 
Training Act, Community Action (paid nonprofessionals) 
and College Work Study - the author attempts to assess 
the direct effects of federal manpower programs in 
reducing unemployment. The three direct effects of 
manpower programs analyzed are: (1) the reduction of 
unemployment caused by the employment in the manpower 
programs of persons previously unemployed; (2) the 
employment of other unemployed persons in jobs left 
vacant by enrollees in manpower programs; and (3) the 
increase in the labor force due to the increased 
opportunities afforded by the manpower programs. 


Conley, Ronald W. 
1965 The Economics of Vocational Rehabilitation. Baltimore: 
The John Hopkins Press. 


A study investigating selected aspects of the economic 
problems caused by physical and mental disability as 
well as the economic benefits of vocational rehabil- 
LiaLion. 


Chapter 4 attempts to evaluate the economic effect- 

iveness of vocational rehabilitation programs in 

the United States, concentrating on the question of 

whether the costs of the program are exceeded by the 
returns. 


Since other data on rehabilitation are sparse, the 
author concentrates on evaluating the increase in 
earnings due to rehabilitation and comparing this 
increase with program costs - indicating the 
statistical shortcomings of this procedure by pointing 
out ways in which program costs may diverge from the 
incremental social costs of rehabilitation service, 

and ways in which the difference in earnings before and 
after rehabilitation services may diverge from what the 
difference in earnings would be if they could be 
measured both with and without services. 
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1969 "A benefit-cost analysis of the vocational rehabilitation 
program.'' Journal of Human Resources 4 (Spring): 226 - 252. 


An examination of the state-federal vocational 
rehabilitation program indicates that funds expended 
on vocational rehabilitation are being repaid, on the 
average, many times over in increased output. In 

fact the actual return on vocational rehabilitation 
tends to be understated because: (1) we have taken a 
consistently conservative approach to estimating the 
future earnings of rehabilitants; (2) we have omitted 
the value of increased homemaking services provided by 
rehabilitants and other unpaid work; and (3) important 
psychic values of becoming a contributing and 
independent member of society are usually overlooked. 


CorazZ iti anh Ghul: 
1967 "When should vocational education begin (a cost-benefit 
analysis).'' From Cathleen Quirk and Carol Sheehan (eds.), 
Research in Vocational and Technical Education. Madison, 
Wisconsin: Center for Studies in Vocational and Technical 
Education, University of Wisconsin. 


A comparison of the benefits and costs of high school 
vocational training as opposed to the benefits and 
costs of post high school vocational training. 


Cornelsen, Leroy A. 
1964 Economics of Training the Unemployed. United States 


Department of Health, Education, and Welfare: Division 
of Vocational and Technical Education. 


The author believes that training of the unemployed is 
a sound economic investment for society. On the basis 
of information on the actual training costs and 
estimated earnings for 12,696 MDTA trainees, he finds 
that: ''Training under the Act has made only modest 
inroads toward alleviating unemployment, but already 
it has demonstrated vividly the economic value of 
training as an investment in our human resources.'' 


Craig, Judith S. (ed.) 

1965 Conference on Follow-up Studies in Educational Research. 
Madison: Center for Studies in Vocational and Technical 
Education, Industrial Relations Institute, University 
of Wisconsin, November, 1965. 


Be 


Proceedings of a conference concerned with the 
objectives and methodology of follow-up surveys 
involved in vocational education evaluation. 


Cushen, W. Edward. 
1964 "A national management worksheet: an aid to strategic 
planning.'' Unpublished paper read at the Canadian 
Operations Research Society and Operations Research 
Society of America Joint Conference, Montreal, 


May 24, 1964. 


A proposed scheme for cost-benefit analysis on a 
macro-level. 


Daniel Yankelovich, Inc. 
1967 The Community Action Programs: Summary of a Research 
Report. United States, Office of Economic Opportunity, 
April, 1967 (Lithographed) . 


A comment on 13 Community Action Programs providing 
compensatory education for children and teenagers, 
Neighborhood Centers, household management, legal 
services, health services, job training and placement 
and adult education. 


1967 CAP Programs and Their Evaluation: A Management Report. 
United States, Office of Economic Opportunity, September, 
1967 (Lithographed). 


A discussion of the Community Action Programs in four 
parts: (1) a summary of the accomplishments of OE0 to 
date; (2) a discussion of CAP's objectives and policies; 
(3) a comment on other major problems confronting 0E0; 
and (4) a section describing the need for evaluation of 
OEO programs as well as some suggested methods for doing 
such evaluation. 


Decore, A.M., M.P. Bourgette and K.M. Lederer. 
1969 The Employment Opportunities Program ||. Edmonton: Alberta 
Human Resources Research Council (Mimeographed) . 


A follow-up evaluation of the Employment Opportunities 

Program operated by the Department of Social Development 
in Alberta. The researchers found that while E.0.P. was 
successful in placing welfare recipients in employment or 
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training, it was not capable of ensuring continuance in these 
involvements over time. Because of the many problems endemic 
to the design and implimentation of this program, all findings 
are questionable and, at best, inconclusive. 


Durkin, Roderick. 
1967 Breaking the Poverty Cycle: A Strategy and Its Eval- 
uation. Cambridge: Harvard University (Mimeographed) . 


The report outlines a procedure for the evaluation of 
Sage Hill Camp -- a small, non-profit work camp in 
Colorado designed to help break the poverty cycle by 
removing teenage boys from a perpetuation milieu and 
introducing the influences of a non-poverty milieu. 


Dymond, William R. 
1969 The role of benefit/cost analysis in formulating man- 
power policy. Pp. 42 - 55 in G. G. Somers and W. D. 
Woods (eds.), Cost-Benefit Analysis of Manpower Policies: 
the Proceedings of a North American Conference. Kingston, 
Ontario: Industrial Relations Centre, Queen's University. 


Benefit-cost analysis has emerged as a most significant 
and useful tool in the appraisal of alternative courses 
of action and alternative resource uses in government 
programs. While cost-benefit analysis may serve as a 
rigorous analytical aid to administrative decision- 
making, it has some substantial limitations partic- 
ularly with regard to the collective or social respon- 
sibilities of government as a basis for the revision of 
human resource development programs. 


Edgerton, Harold A. 
1967 'The prediction and outcomes of MDTA programs.'' Pp. 17 - 26 
in Cathleen Quirk and Carol Sheehan (eds.), Research in 
Vocational and Technical Education. Madison, Wisconsin: 
Center for Studies in Vocational and Technical Education, 
University of Wisconsin. 


Using data from MDTA programs collected by the Office 
of Manpower Policy, Evaluation and Research, as well 
as a questionnaire answered by program directors, the 


ails 


author describes some program characteristics and 
actions essential to the minimization of dropouts. 


Evans, John W. 


1969 "Evaluating social action programs.'' Social Science 
Quarterly 50 (December): 568 - 581. 


The author maintains that there are several reasons for 
the lack of objective, empirical evaluation: the 
absence of an adequate social science methodology, the 
relative primitiveness of our present measuring 
instruments, the fact that the real world environment 
is too complex to sort out casual relationships, that 
empirical evaluation is a foreign way of thinking to 
government administrators and program directors, and 
that the demands of program administration and 
objective evaluation are often mutually exclusive. A 
plan for the evaluation of antipoverty programs has 
been initiated by the OE£0O in which evaluations are 
categorized as follows: (1) assessment of overal] 
program impact and effectiveness to determine success 
in achieving basic objectives; (2) evaluation of the 
relative effectiveness of different program strategies 
to determine alternative techniques; and (3) eval- 
uation of individual projects through site visits 

and other monitoring activities to assess managerial 
and operational efficiency. 


Ferman, Louis A. and Scott Harvey. 


1968 


“Joberebratningsin Michiiganad* ps r2lijhe 2S 5eingGerald G. 
Somers (ed.), Retraining the Unemployed, Madison, Wisconsin: 
University of Wisconsin Press. 


A critical examination of a number of training and 
retraining efforts in Michigan to see what contri- 
butions they made to the functioning of the state 
economy as well as to the adjustment of workers to the 
changing world of work. In a broader sense, the 
authors are interested in determining what lessons can 
be learned from the Michigan experience with retraining 
programs that can be applied to national economic 
problems. 


The report reviews the problems and needs of the 
Michigan economy which have made special programs of 
training and retraining necessary and then focuses 
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on the operation of the two major retraining 

programs sponsored by the federal government: 
the Area Redevelopment Administration and the 
Manpower Development and Training Act. 


Gerard, Robert A. 
1966 Summer Community Youth Work Program, Final Report. 
Pittsburgh: Office of Manpower Policy, Evaluation 
and Research, Action-Housing Inc., September, 1966. 


An evaluation of a nine-week demonstration project for 
16-21 year-olds held in Pittsburgh, Pennsylvania during 
the Summer of 1966. 


Gibbard, Harold A. and Gerald G. Somers. 

1968 "Government retraining of the unemployed in West Virginia.'"' 
Pp. 17 - 124 in Gerald G. Somers (ed.), Retraining the 
Unemployed. Madison, Wisconsin: University of Wisconsin 
Press. 


This study attempts to describe and evaluate the 
effectiveness of training for selected occupations in 
five counties in West Virginia. The key facets of the 
study were: (1) An examination of the retraining 
courses themselves: their legislative basis, 

variables affecting the scope of retraining programs in 
different localities, the administrative procedures by 
which courses were set up and trainees selected, and 

the manner in which the retraining courses were 
conducted; (2) An analysis of the attitudes and 
characteristics of the men and women selected for 
training: comparisons with the men and women who 

applied for training and were not accepted, and with 
those who were unemployed when retraining courses were 
open, yet did not apply for retraining; (3) A review 

of the outcome of retraining: the employment experiences, 
attitudes, mobility of men and women subsequent to 
retraining; the feelings of employers toward the 
retraining programs and toward the retrained workers whom 
they hired, a comparison of how retrained men and women 
fared against men and women who were unemployed when 
retraining courses were open but who did not get retrain- 
ing; and (4) the gains and costs of the retraining 
programs to the individuals and to society, including 

an appraisal of the impact of retraining programs on the 
levels of unemployment in the areas where the retraining 
programs were conducted. 


ya 


Glickstein, Howard A. 
1969 "Federal educational programs and minority groups.'' 
Journal of Negro Education 38 (Summer): 303 - 314. 


The author claims that program evaluation of efforts 
to provide equal educational opportunities for 
minority groups is minimal or nonexistent. Similarly, 
current programs lack proper coordination and follow- 
up, and several successful programs have not received 
adequate appropriations. 


Goldfarb, Robert S. 
1969 "The evaluation of government programs: The case of 


New Haven's manpower training activities.'' Yale Economic 
Essays 9 (Fall): 59 - 104. 


This paper provides information on the operation 
of existing manpower training programs and on the 
implications that this program experience has for 
developing improved training programs. The author 
also presents an excellent criticism of current 
benefit-cost techniques, stressing the various 
methodological deficiencies inherent in present 
manpower program evaluation. 


The deficiencies of cost-benefit analyses investigated 
are (1) the nonmeasurability and nonpriceability of 
certain benefits and costs; (2) the general failure to 
assess the relative efficiency of program components 

so as to maximize resource allocation and make optimal 
use of the ''learning potential'', and (3) the control 
group problem. 


Using two types of manpower programs - institutional or 
classroom training and government-sponsored on-the-job 
training - operating in New Haven from 1964-1966, the 
author demonstrates the methodological problems of 
program evaluation. 


The author argues (in light of the New Haven exper- 
jence) that the achievement of the desired goals 

of training programs may not always require training 
itself. 


Gordon, Jesse E. 
'Testing, counselling and supportive services for dis- 


advantaged youth: experiences of MDTA experimental and 
demonstration projects for disadvantaged youth.'"' 
Unpublished manuscript. 


A report attempting to identify new knowledge and techniques 
growing out of experiments, in working with disadvantaged 


youths. 
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Gordon, Margaret S. 
1964 "Retraining programs - at home and abroad.'' Pp. 128 - 
138 in Proceedings of the Seventeenth Annual Meeting, 
Industrial Relations Research Association, December, 
1964. Madison, Wisconsin: IRRA. 


A discussion of the European experience in government 
retraining programs and a comment on the implications 
this has for manpower policies at home. 


Green, Alfred L. 

1963 'A study and appraisal of manpower programmes as related 
to a policy of full employment in France, Great Britain, 
the Netherlands and Sweden.'' Paper prepared for the 
New York State Department of Labor, September, 1963. 


A general discussion of retraining programmes in 
Western Europe with an attempt to assess their impact 
upon problems of unemployment and technological 
displacement. 


Hansen, W. Lee, et al. 
1967 Determinants of Earnings: Does Schooling Really Count? 
Madison: University of Wisconsin Press, December, 1967. 


An attempt to demonstrate that vocational retraining 
might well bring about greater improvements in income 
than general education, especially for the disadvantaged. 


Hardin, Einar 
1969 Benefit-cost analyses of occupational training programs: 
a comparison of recent studies. Pp. 97 - 119 in G.G. 
Somers and W.D. Woods (eds.), Cost-Benefit Analysis of 
Manpower Policies: the Proceedings of a North American 
Conference. Kingston, Ontario: Industrial Relations 
Centre, Queen's University. 


Benefit-cost analysis of occupational training may in 
principle be undertaken from at least three points of 
view: society as a whole, the individual trainee, and 
the government as an organization. Using studies of 
the strictly economic costs and benefits of retraining, 
the author considers in turn the concepts used in 
relation to each of these points of view. Comparisons 
are made in terms of the: (1) definition of concepts; 
(2) control group usage; (3) the measurement of the 
differential effects of training; (4) observation 
periods and time trends; (5) statistical methods and 
data utilized; and (6) findings. 


Se 


Herman, Melvin and Michael Munk. 
1968 Decision Making in Poverty Programs: Case Studies from 
Youth-Work Agencies. New York: Columbia University Press. 


A collection of case studies of administrative and 
operational decision-making in poverty programs with 
a section on the problems of research and evaluation. 


Hernstadt... | nwimeL....weteal.. 

1969 The impact of federal manpower programs on the city of 
Boston.'' Paper prepared for the Allied Social Science 
Association Meetings, Industrial Relations Research 
Association, New York, December 29, 1969. 


A discussion of the various agencies involved in the 
administration of federally sponsored manpower programs 
in Boston and an evaluation of the effectiveness with 
which these agencies implement such manpower programs. 


Hess, ‘Robert._D. and June L. Tapp. 
1967 An Evaluation of the Effectiveness of a Community-based 
Manpower Training Program. Chicago: University of Chicago. 


An assessment of a manpower training study conducted 
by the Woodlawn Organization (TWO) in the Negro 
Woodlawn area of Chicago. 


Hoffman, N. March. 
1967 Follow-up Study of MDTA E & D Project at Bluefield State 
College. United States: Bureau of Social Science 
Research, Inc., May, 1967. 


A description and evaluation of the experience and 

results of a program in skill training, basic education 
and job placement sponsored by Bluefield State College 

in Bluefield, West Virginia. An evaluation by a BSSR 
study team revealed that there is little evidence to 
suggest that the training program had an impact on 
increasing the incidence of employment or lowering welfare 
participation. Several possible reasons for this failure 
are proposed. 


Hoos, Ida R. 
1967 Retraining the Work Force. Berkeley and Los Angeles: 
University of California Press. 


The author presents a series of case studies of a 
sample of training programs in the United States 
carried out by (1) adult vocational schools; 

(2) civil service; (3) private companies; 
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(4) labor unions; and (5) the federal government. 
Data about a wide variety of programs was collected 
through participant observation, consultation with 
teachers and officers, interviews with trainees, 
perusal of records and personnel, telephone and mai | 
follow-ups on dropouts and graduates. Case histories 
of some trainees are also included. 


Human Interaction Research Institute. 


1967 An Evaluation Follow-up Study of the Job Opportunity 
Center Experimental and Demonstration Project for the 
Communities of Denver and Pueblo, Colorado: Final 
Report. Los Angeles: Human Interaction Research 
Institute, September, 1967. 


An assessment of the Job Opportunity Center (JOC) 

which was to provide an intensive program cf recruitment, 
selection, counselling, testing, referral to training 

and placement for unemployed heads of families who were 
unable te profit from conventional programs. The study 
showed that the administration failed to provide adequate 
staff leadership, to clarify objectives, to execute 
sufficient advance planning to ensure program support, 

or to secure essential cooperative working relation- 
ships with other agencies. 


Industrial Relations Research Association. 
1964 Industrial Relations Research Association's Proceedings 


of the Seventeenth Annual Meeting. Gerald G. Somers 
(ed.). Chicago, Illinois, December 28 - 29, 1964. 


Proceedings of the Industrial Relations Research Assoc- 
iation's Seventeenth Annual Meeting emphasizing predic- 
tions on the future course of industrial relations as 
well as a description and analysis of current issues 

at home and abroad. 


1967 The Development and Use of Manpower, the Industrial 
Relations Research Association's Proceedings of the 
Twentieth Annual Winter Meeting. Washington, D.C.: 
December 28 - 29, 1967. 


Proceedings of the Industrial Relations Research Assoc- 
iation's Twentieth Annual Winter Meeting devoted to 
papers and discussions on the ''Development and Use of 
Manpower'', 
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The Institute for Local Self-Government. 
1969 A New Careers Evaluation. Berkeley: The Institute 
for Local Self-Government and the Oakland Redevelop- 
ment Agency. 


The study attempts to assess the effectiveness of the 
Oakland Redevelopment Agency's New Careers Program. 
Designed to serve as a model for measuring the 
accomplishments of individual and systematic change, 
the study demonstrated that the New Careers program can 
solve manpower needs through employment of the 
disadvantaged. 


Jenness, Robert A. 
1969 "Manpower mobility programs.'' Pp. 119 - 151 in G.G. Somers 
and W.D. Wood (eds.), Cost-Benefit Analysis of Manpower 
Policies, the Proceedings of a North American Conference. 
Kingston, Ontario: Industrial Relations Centre, Queen's 
University. 


The article presents a cost benefit model for manpower 
mobility programs. 


Johnson, Louise A. 
1967 Follow-Up Study of MDTA E & D Project Conducted by 
Tuskegee Institute. United States: Department of 
Labor, Bureau of Social Science Research, Inc., 


May, 1967. 


A follow-up report of an MDTA Experimental and Demon- 
stration project conducted by Tuskegee Institute 

for 180 Negro male residents of rural Alabama. 
Evaluation by a Bureau of Social Science Research 
study team revealed that the program was successful 
in providing a majority of the men interviewed with 

a new occupation. 


1967 Follow-Up Study of MDTA E & D Project Conducted at 
Agricultural and Industrial State University at 
Nashville. United States: Department of Labor, 
Bureau of Social Science Research, Inc., July, 1967. 


A follow-up report of an MDTA Experimental and Demon- 
stration project for 180 adults, men and women, Negro 
and white, predominantly from three rural areas of 
Tennessee. An evaluation by a BSSR study team found 
that while the program of basic education and job 
training was beneficial, actual job placement was 
generally inadequate. 
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Judy, Richard W. 
1969 "Costs: theoretical and methodological issues.'' 
Pp. 16 - 29 in G.G. Somers and W. D. Wood (eds.), 
Cost-Benefit Analysis of Manpower Policies, the Pro- 
ceedings of a North American Conference. Kingston, 
Ontario: Industrial Relations Centre, Queen's 
University. 


A discussion of conceptual and methodological questions 
surrounding the definition and measurement of costs in 
cost-benefit analysis. The author emphasizes certain 
problems of cost measurement that are often ignored in 
actual practice and points out that cost-benefit 
analysis appears to be a fairly crude tool until it is 
compared with the available alternatives. 


Kaufman, Jacob J. 
"An experimental program to compare education versus 
training for young school dropouts.'' United States: 
Office of Education, Bureau of Research (Mimeographed). 


An investigation of the economic and psychological 
effects of providing high school dropouts with either 
extended training for a specific occupation, or with 
the opportunity to complete a high school diploma. 
Comparisons of these two groups will be made in this 
study covering July, 1965 - March, 1970. 


Kershaw, Joseph. 
1966 Systems Analysis and the War on Poverty. Proceedings 
of the Joint Meeting of the American Society for 
Training and Development and the Society for Personnel 
Administration, February, 1966. 


Uses systems analysis to evaluate the relative costs 
and benefits of governmental programs. 


Kirby, Frederick C. and Paul A. Castagna. 
1969 Benefit-Cost Analysis of TAT Phase 1 Worker Training. 
Knoxville, Tennessee, Department of Economics. 


An analysis of the benefits and costs of the Training 
and Technology (TAT) Project conducted at a large 
industrial plant in Oak Ridge, Tennessee. TAT, a 
continuing experimental program in training skilled 
workers for private industry, was found to have a high 
rate of return to the individual trainee. 
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Kraft, Ivor. 
1966 Head start to what?'' The Nation 203 (September). 


The author argues that there is very little substantial 
research evidence to support the alleged achievements of 
Project Head Start. He maintains that this is a case 
where a potentially good program has been ''over-sold!! 
for the wrong reasons - that preschooling can never be 

a major antidote for poverty as it is too far removed 
from the root causes. 


Levine, Marvin. 
1963 An evaluation of retraining programs for unemployed 
workers in the United States. Unpublished doctoral 
dissertation, University of Wisconsin. 


A report examining the effectiveness of public and 
private retraining efforts, with a case study of a 
group of trainees and nontrainees in a redevelopment 
area. 


Levine, Robert A. 
1968 "Rethinking our social strategies.'' The Public Interest 
10 (Winter): 86 - 96. 


This author argues that the highly administrative 
approaches to the planning of manpower, welfare, and 
programs of other types, whether centralized or decent- 
ralized, do not work or only work on a small scale 
compared to the size of the problem. He would arque 
that we have to provide more incentives for individuals 
to develop their own programs. 


1969 'Manpower programs in the war on poverty.'' Pp. 170 - 183 
in G.G. Somers and W.D. Wood (eds.), Cost-Benefit Analysis 
of Manpower Policies, the Proceedings of a North American 
Conference. Kingston, Ontario: Industrial Relations 
Centre, Queen's University. 


The author expresses some skepticism over the unrest- 
ricted use of cost-benefit analysis as a major policy 
determinant. By referring to the use of cost-benefit 
criteria in the Job Corps, the author argues that 

certain reservations are necessary due to the fact that: 
(1) when talking about cost-benefit analysis of manpower 
programs, and particularly in manpower programs for youth, 
we are speaking of a benefit measured by lifetime earnings, 
but we are estimating those benefits from the first month 
of so of the post-program experience; (Zit is virtually 
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impossible to randomize in Federal programs with the 
net result that the typical control group consists 

of those persons who signed up for an appointment but 
who did not show up; and (3) a multiplicity of differ- 
ences makes the cross-comparison of such programs 
extremely difficult and of rather dubious value. 


Levitan, Sar A. 
1964 Federal Manpower Policies and Programs to Combat 
Unemployment. Kalamazoo, Michigan: W.E. Upjohn 
Institute for Employment Research, February, 1964. 


A rather general discussion of the impact of federal 
manpower policies and programs in reducing unemployment. 


1966 "An evaluation of the social security system as an 
income protection program.'' Pension and Welfare News 
2 (September). 


An evaluation of various income maintenance programs 
for the elderly. 


1967 Antipoverty Work and Training Efforts: Goals and 
Reality. Policy Paper in Human Resources and Industrial 
Relations, No. 3 Ann Arbor, Michigan: Institute of 
Labor and Industrial Relations, the University of 
Michigan - Wayne State University, with the National 
Manpower Policy Task Force. 


A discussion of several programs (Job Corps, 
Neighborhood Youth Corps, and Work Experience and 
Training) inaugurated under the Economic Opportunity 
Act, stressing job creation and training. 


1967 "Head Start: It is never too early to fight poverty.' 
December 10, 1967 (Mimeographed) . 


The author criticizes the Administration and the Office 
of Economic Opportunity for playing politics with the 
Head Start program and for attempting to sell the 
program as an educational panacea. It would be more 
realistic for Head Start to function as a preschool 
program for primary schools offering poor children an 
enriched and more comprehensive education. 
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1968 "Manpower aspects of the economic opportunity act.!! 


pp. 172 - 181 in Gerald G. Somers (ed.), The Development 


and Use of Manpower. Washington: Industrial Relations 
Research Association. 


The author discusses the working and training programs 
under the Economic Opportunities Act and questions 
whether they are achieving the goals established by 
legislation. 


1969 Programs in Aid of the Poor for the 1970's, Policy 
Studies in Employment and Welfare No. 1. Baltimore: 
Johns Hopkins Press 


This study surveys current aid programs and suggests 
future priorities if eradication of poverty is to succeed. 


Levitan Sar A. and Garth L. Magnum. 
1967 ''Making sense of federal manpower policy.'' Policy 
paper in Human Resources and Industrial Relations No. 
Ann Arbor, Michigan: Institute of Labor and Industrial 
Relations, the University of Michigan - Wayne State 
and the National Manpower Policy Task Force. 


A descriptive summary of federally supported Manpower 


Programs, emphasizing the problems arising from 
inadequate coordination. 


Main, Earl D. 


1968 "A nationwide evaluation of MDTA institutional job 
training.'' Journal of Human Resources 3 (Spring). 
Loon / Oe 


A study of 1200 former MDTA trainees and 1060 other 
persons who were unemployed about the same time the 
training courses started revealed that the MDTA program 
had no demonstrable effect on income. Both trainees 
and non-trainees reported about the same weekly wages 
on their most recent full-time jobs; however, trainees 
were noted to have a significant advantage in terms of 
increased employment. 


Magnum, Garth L. 
1967 The Contributions and Costs of Manpower Development 


and Training. Policy Paper in Human Relations, No. 5 
Ann Arbor, Michigan: Institute of Labor and Indust- 
rial Relations, Michigan - Wayne State University and 
the National Manpower Policy Task Force. 
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A general evaluation of the objectives, costs and 
issues involved in the Manpower Development and 
Training Act. The author feels that the evidence 
accumulated in cost-benefit studies is conclusive 
enough to leave little doubt that the program is 
a good economic investment. 


1967 'The emergence of a national manpower program.'' Pp. 11 
- 40 in Robert A. Gordon (ed.), Toward a Manpower Policy. 
New York: John Wiley and Sons. 


The author assesses the contributions of present man- 
power programs - claiming that as solutions to general 
unemployment they have little to offer, but as solutions 
to the problems of individuals competitively disadvant- 
aged in favorable labor markets, their contributions have 
been and can be significant. 


1967 "Evaluating vocational education: problems and 
priorities.'' Paper presented at the Stanford 
Research Institute Conference on Vocational - 
Technical Education: New Approaches to Planning, 
Programming, Budgeting and Evaluation, April 10 - 12, 
1967. 


This paper discusses vocational education in the United 
States, contrasting what it presently is with what it 
should be if it is to be successful. The author 
challenges the very assumptions and goals of the 
Vocational Education Act of 1963 and endeavors to 
assess alternative methods of occupational prepar- 
ation in terms of comparative opportunity costs. 


1968 a ''Evaluating manpower programs.'' Monthly Labor Review 
91 (February): 21 = 22. 


Because little attention has been given to developing 
policies and procedures for effective program 
evaluation, an adequate assessment of the programs 

and practices comprising the federal manpower policy is 
lacking or of) limitedsutility: 


1968 b "Evaluating federal manpower programs.'' Pp. 161 - 171 
in Gerald G. Somers (ed.), The Development and Use of 
Manpower. Washington: Industrial Relations Research 
Association. 


This report is concerned with the Manpower Development 
and Training Act, the Voaational Education Act of 1963, 
the Vocational Rehabilitation Program, and the U.S. 
Employment Service during the period 1961-67. While 
some programs have proven useful in lowering obstacles 
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to employment and increasing job retention of the 
disadvantaged, most suffer from serious shortcomings: 
(1) the array of programs does not represent a 
systematic interrelated effort to identify and provide 
each of the services required by the disadvantaged; 
(2) the enrollments and resources in the programs are 
too small relative to the need; (3) budgetary 
commitments are not rationally related to need; 

(4) the administrative capacity to deliver services 
has not yet been developed; there are not adequate 
data for evaluation of strengths and weaknesses. 


1968 c MDTA: Foundation of Federal Manpower Policy. Baltimore: 
The John Hopkins Press. 


The author attempts to trace the development of 
various policy decisions by which MDTA has assumed its 
present shape, and to evaluate the successes and 
failures, benefits and limitations, of the MDTA. 
Evaluation of the MDTA is made by examining its impact 
on the employment and earnings of its participants, 
supplemented by the assessment of certain other 
nonquantifiable contributions. 


1968 d ''Manpower programs U.S.A.: an inventory.'! New 
Generation 50 (Winter): 18 - 21. 


A criticism of the lack of composite coordination among 
national manpower programs which encourages the 
interagency competition for funds. Faced with the 
conflict between its new job of helping the disadvant- 
aged and its traditional job of upgrading the labor 
force by providing the best job applicant, some federal- 
state employment services (e.g. vocational educators) 
are ignoring the 1963 Congressional instruction to give 
greater attention to the disadvantaged. 


Mangum, Garth L. and L.M. Glenn. 
1967 "Vocational rehabilitation and federal manpower policy." 
Policy paper in Human Relations and Industrial Relations 
No. 4. Ann Arbor, Michigan: Institute of Labor and 
Industrial Relations, the University of Michigan - 
Wayne State University and the National Manpower Policy 
Task Force. 


The authors feel that the Vocational Rehabilitation 
reporting system is reasonably adequate for managing a 
rehabilitation program but there is no follow-up 
information to allow realistic assessment of program 
results beyond immediate employment. 
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Manpower Evaluation Report #1. 
1963 "Training disadvantaged groups under manpower development 
and training act.'' Paper prepared by Chester W. Hepler, 
November, 1963. United States Department of Labor, Washington, 
ee 


This report attempts to evaluate the extent to which 
training programs established under the Manpower 
Development and Training Act are providing the 
necessary marketable skills to persons who, because of 
certain personal characteristics or lack of suitable 
work skills, are markedly disadvantaged in the 
competition for available jobs. 


Manpower Evaluation Report #3. 
1964 "Occupational training of women under manpower develop- 
ment and training act.'' Paper prepared by Helen Wood, 
Jeanette Siegel and Benjamin Goldstein, July, 1964. 
United States Department of Labor, Washington, D.C. 


This report endeavors to evaluate the degree to which 
training programs established under the Manpower 
Development and Training Act are really meeting the 
needs of unemployed women. 


Manpower Evaluation Report #4. 
1964 ''MDTA training programs: comparison of 1963 and 1964."! 
Paper prepared by Benjamin Goldstein, December, 1964. 
United States Department of Labor, Washington, D.C. 


This report compares Manpower Development and Training 

Act institutional training programs for the fiscal year 
1964 with those for 1963, assessing the various program 
changes and developments. 


Manpower Evaluation Report #5 
1965 "Graduates of Norfolk Project one year later.'' Paper 
prepared by Pierce A. Quinlan, October, 1965. 
United States Department of Labor, Washington, D.C. 


An evaluation of the demonstration retraining project 
at Virginia State College (Norfolk Division) one year 
after the completion of training, revealed that 99 
percent of the graduates had found employment during 
the year (as compared to the 72 percent national 
average for MDTA institutional trainees). Several 
possible reasons for the success of trainees are 
suggested. 
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Manpower Evaluation Report #7. 
1967 "Earnings mobility of MDTA trainees.'' Paper prepared by 
Rose G. Rosofsky and Ray W. Kohen, April, 1967. United 
States Department of Labor, Washington, D.C. 


A study of over 12,000 trainees who completed MDTA 
institutional courses during the first half of 1965 
demonstrates the effectiveness of Manpower Development 
and Training Act training in upgrading skills and 
enlarging the earning capacities of trainees. 


Manpower Evaluation Report #8. 
1968 ''The influence of MDTA training on earnings.'' Paper 
prepared by Rose G. Rosofsky and Benjamin Goldstein, 
December, 1968. United States Department of Labor, 
Washington, D.C. 


This study examines, as did its predecessor (Manpower Evaluation 
Report No. 7), the effectiveness of the MDTA training program 

in upgrading its graduates and, in so doing, enhancing their 
earning capacities. As in the case of the initial pilot 

sample, the considerable volume of data studied here demon- 
strated a general upward shift in overall] hourly earnings 
following training (as compared with pretraining earning 
levels), but with differing impact upon various component 
groups. 


Matteson, Gerald. 
1965 "Vocational training and employment opportunities for 
rural youth.'' Annual Report. Madison, Wisconsin: 
Center for Studies in Vocational and Technical 
Education, University of Wisconsin. 


A study planned to evaluate high school vocational 
education for selected male graduates in their 
post-high school employment. 


McKechnie, Graeme. 
1966 "Retraining and geographic mobility: an evaluation." 
Unpublished Ph.D. dissertation, University of Wisconsin. 


The author notes that retraining when combined with 
geographic mobility enhances income beyond the level 
attained by either retraining or mobility alone. 


Miller, S.M. and Martin Rein. 
1965 The war on poverty: perspectives and prospects. 
Pp. 272 - 320 in Ben B. Seligman (ed.), Poverty as a 
Public Issue. New York: The Free Press. 


ca 


The authors press for a strategy of analysis and 
change with regard to the War on Poverty. Since 
there is no guarantee on how far or how fast this 
program will move to alleviate the plight of the 
poor, informed social analysis and increased 
political pressure will be needed to move the 
poverty campaign into more effective channels. 


Miller, S.M., Pamela Roby and Alwine A. de Vos van Steenwi jk. 
1970 "'Creaming the Poor.'' Transaction (June); 38 - 45. 


Analyzes programs aimed at improving the conditions 
of the poor and argues that efforts to improve the 
conditions of the poor, when effective, result in 
improving those at the top of the bottom, leaving 

the worst deprived possibly worse off than before. 
Selection and exclusion of potential recipients 

occur at several points in the contact between 

agency and poor client. People must jump the 
obstacles in order to obtain delivery of services. 

An individual may be excluded from a program by his 
ignorance about its existence, by being turned off 

by the impression it gives him, or by finding it 
physically inaccessible. Admittance criteria or 
usage of referral systems weed out other clients. 
Successful completion of programs is dependent upon 
the understanding and openness of agencies to various 
modes of individual behavior. This process of creaming 
by agencies which serves to reduce risk, to make a 
record and to help those most likely to benefit from 
assistance, all work against the objective of dealing 
with the poorest. 


Mincer, Jacob. 
1962 "On-the-job training: costs, returns and some implications.'' 
Journal of Political Economy, Supplement LXX (October). 


An article concerned specifically with the investment 
aspect of on-the-job training. 


Mooney, Joseph D. 
1966 "Teenage labor problems and the neighborhood Youth 
Corps.'' Pp. 95 - 120 in Frederick H. Harbison and 
Joseph D. Mooney (eds.), Critical Issues in Unemploy- 
ment. Princeton: Princeton University. 


A quantitative profile of teenage labor problems with a 
preliminary evaluation of the Neighborhood Youth Corps 
(based on the experiences in Newark and Trenton, New Jersey). 
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Morrison, James W. 
1966 a ''A comprehensive evaluation model for community action 
programs."" Paper prepared for the 1966 Annual Meeting of 


the Society for the Study of Social Problems, Miami Beach, 
Florida, August, 1966. 


A description of the use of several kinds of evaluative 
research in studying the effectiveness of Community Action 
Programs. 


1966 b ''Designs and models for community action program eval- 
uation.'' Paper prepared for the 1966 Annual Meeting of 
the Society for the Study of Social Problems. Miami Beach, 
Florida, August, 1966. 


An examination of the Community Action Agencies 
(CAA) for the purpose of determining optimal 
service efficiency. 


Moser, Collette 
"An evaluation of area skill surveys as a basis for 
manpower policy.'' Doctoral dissertation in process, 
University of Wisconsin. 


Attempts to evaluate Area Skill Surveys (local labor 
market studies which project detailed occupational 
supply and demand for 2 to 5 years) in terms of their 
accuracy and implementation. 


Myers, Albert E. 
1970 'The impact of evaluative research on educational 
programs for the poor.'' Record 71 (February). 


Because of a number of considerations, the author 
feels that much top grade scientific research is of 
questionable value. Thus, while there is a 
proliferation of evaluative reports on federally 
financed programs, very little of the eventual 
decision making is actually based on these efforts. 


Nosow, Sigmund. 
1968 Retraining Under the Manpower Development Training Act: 
A Study of Attributes of Trainees Associated with 
Successful Retraining. East Lansing, Michigan: School 
of Labor and Industrial Relations, Michigan State 
University, January, 1968. 


A report on two related studies: (a) examines the 

personal and social characteristics of those who enter 
retraining and investigates the labor market outcomes 

of retraining three months after course completion; 

and (b) compares persons who enter retraining with a con- 
trol group in order to assess the consequences of retraining. 
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Page, David A. 

1964 "Retraining under the manpower development act: a cost- 
benefit analysis.'' Pp. 257 - 276 in John D. Montgomery 
and Arthur Smithies (eds.), Public Policy. Cambridge: 
Graduate School of Public Administration, Harvard Univ- 
LB AR aS) 


The author feels that there is accumulating evidence to 
suggest that the government retraining of the unemploy- 
ed is a sound public investment - at least, in terms of 
costs and benefits. 


Palomba, Neil A. 
1968 "Evaluation of the lowa governor's dropout program.''! 
lowa Business Digest 40 (October). 


An examination of the Governor's Dropout Program in 

lowa whereby student dropouts were given advice on a 

job, manpower training, the Neighborhood Youth Corps, the 
the Job Corps, or other means of completing their 
education and developing marketable skills. The 

article focuses on the effectiveness and costs of the 
G.D.P. during the period March, 1967 through June, 

1967. 


Peterson, Gertrude D. 

1965 "An evaluation of the concept of trainee camps for 
unemployed youth.'' Paper prepared for the United States 
Department of Labor, Manpower Administration. Menlo 
Park, California: Stanford Research Institute, 

June, 1965. 


An assessment of the goals, strengths and weaknesses 
of the Oak Glen Youth Camp for unemployed youth. 


Polianski, A.N. 

1967 ''Manpower mobility program - a pilot project in the 
method of evaluation of government programs.'' Paper 
prepared for the 25th Interstate (1st North American) 
Conference on Labor Statistics, Toronto, June 15, 1967. 


This paper presents a discussion of the Manpower 
Mobility Program, a cost/benefit analysis of the 
program (with special emphasis on the problems 
involved in such an undertaking), and an evaluation 
of the findings of the cost/benefit analysis to see 
if they contain any possible policy implications. 
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The primary conclusions are that for policy purposes 
individuals! experience should not be extrapolated to 
aggregate levels and that cost/benefit ratios have to 
be evaluated not only on their own merit but within 
broader frameworks. 


Resource Management Corporation. 
1969 Evaluation of the War on Poverty: The Feasibility of 
Benefit - Cost Analysis -for Manpower Programs. 
Bethesda, Maryland: RMC, March, 1969. 


This paper reports the findings of a study conducted to 
investigate the feasibility of applying benefit-cost 
analysis to anti-poverty programs. In the case of the 
Out-of-School and Job Corps programs (the two 
Neighborhood Youth Corps Programs examined), it was 
noted that there was little conclusive evidence by 
which to assess the economic viability of such efforts. 


Rico, Leonard. 
1965 ''The manpower development and training act: a case 
study.'' The Quarterly Review of Economics and Business 
5 (Summer) . 


A case study of the MDTA programs in Philadelphia 
analyzing the relative effectiveness of the MDTA in 
alleviating hard-core unemployment. 


The author concludes that ''the data leave no doubt 
that the MDTA has not done an impressive job in 
ameliorating the unemployment problem.'' 


Furthermore, if the national policy is to be 
successful, it must relate the MDTA effort to the 
realities of the labor market on both the demand and 
supply sides. 


Schneiderman, Leonard. , . 
1968 "Profectehead startt. aprons te cover their dirty clothes.” 
Mental Hygiene, 52 (January): 34 - 41. 


A national strategy to neutralize the destructive 
effects of slum home conditions upon children is 
unlikely to be effective unless it is able to enrich 
family life with more resources and opportunities. 
Projects Kindergarten and Head Start are necessary In 
bringing public education to all those children who 
need it and in making this education relevant. 
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Scott, Loren, G. 

1970 ''The economic effectiveness of the on-the-job training: 
the experience of the Bureau of Indian Affairs in 
Oklahoma.'' Industrial and Labor Relations Review 23, 
(January): 220 = 236% 


An evaluation of the Bureau of Indian Affairs 

on-the-job training program which was designed to 

provide Indian participants with an opportunity to 
increase their skills through work experience. A 
theoretical framework for benefit-cost analysis of 

the program was developed along the following dimensions: 
(1) private benefits; (2) social benefits; (3) private 
costs; and (4) social costs. Data are also presented 
with regard to completion of training, characteristics 

of the trainees, and productivity differentials. 


Sewell, David O. 
1967 "A critique of cost-benefit analysis of training."' 
Monthly Labor Review 90 (September): 45 - 51. 


Contrary to expectations, there is little in present 
program evaluations to justify optimism about the value 
of educational expenditures in ameliorating poverty and 
its related conditions. The author believes that it is 
totally incorrect to generalize from the benefit-cost 
findings for the programs analyzed (for example, the 
Job Corps) to other training schemes dealing with the 
target population of the War on Poverty. Subsequent 
sections of this paper attempt to document this 
hypothesis and examine other more technical 
shortcomings of the published studies considered as 
benefit-cost analyses of training programs. 


Sharp, Laure M. and Rebecca Krasnegor. 
1966 The Use of Follow-Up Studies in the Evaluation of 
Vocational Education. Washington: Bureau of 
Social Science Research, Inc., May, 1966. 


Follow-up studies of vocational education program 
graduates were found to be a useful tool in the 
evaluation of training. This report presents a 
number of findings and recommendations with regard to 
the use of follow-up studies as well as an annotated 
bibliography summarizing the pertinent publications 
and studies analyzed in connection with this report. 
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Sherwood, Clarence C. 
1966 Guidelines for a Data Collection System for Community 


Programs for Unemployed Youth. Center for the Study of 
Unemployed Youth, February, 1966. 


Discusses the need for more extensive data collection 
and program evaluation for community programs 
involving unemployed youths. 


Silverman, Leslie J. 
1967 a Follow-Up Study of MDTA E & D Project at North Carolina 
A & T State College. Washington, D.C: Bureau of Social 
Science Research, Inc., March, 1967. 


An evaluation of Project Uplift | - an MDTA Experimental 
and Demonstration Project sponsored by A & T College 

of North Carolina and designed to reach, recruit, 

train and place disadvantaged rural adults. The BSSR 
study team who performed the evaluation found the 
program to be ''poorly conceived and conducted''. 


1967 b Follow-Up of Project Uplift, the MDTA E & D Project 
Conducted by Florida A & M University. Washington, D.C.: 
Bureau of Social Science Research, Inc., July, 1967. 


The conclusions of this evaluation are largely nega- 
tive - the project was neither well administered nor 
coordinated, with the result that most placement 
success was confined to traditional Negro jobs, and 
no trainees passed the federal civil service test. 


SOUTER AT COUT Ee Bee bien cll. 
1966 Evaluation and Skill Training of Out-of-School, Hard 
Core Unemployed Youth for Training and Placement. 
St. Louis: Department of Education, St. Louis University. 


A follow-up study of the effectiveness of a youth 
training project administered by the Jewish Employment 
and Vocational Service of St. Louis, Missouri.-~ The 
researchers look at four major areas related to youth 
training - the agency, the trainee, the employer, and 
the trainee's integration into society. 


Solie, Richard J. 
1968 a ''The employment effects of retraining the unemployed." 
Industrial and Labor Relations Review 21 (January): 
21) eee 
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A study attempting to assess the effect of 
government-sponsored retraining programs on the 
labor market experience of the unemployed in 
depressed rural areas. The evidence obtained in 
this and other studies appears to suggest that re- 
training programs do improve the employment exper- 
ience of unemployed workers; however, part of the 
advantage of those who complete training is the 
result of their more favorable socio-demographic 
characteristics. 


1968 b ''An evaluation of the effects of retraining in Tennessee." 
Pp. 193 - 211 in Gerald G. Somers (ed.), Retraining the 
Unemployed. Madison: University of Wisconsin Press. 


The author examines the effect of government- 
sponsored retraining programs on the labor market 
experience of the unemployed in depressed rural 
areas. Using empirical findings from a study of 

an early Area Redevelopment Administration training 
pregram, the following conclusions are derived: 

(1) evidence of this and other studies strongly 
Suggests that retraining programs do improve the 
employment experience of unemployed workers; (2) it 
is not clear that retraining reduces the overal | 
level of unemployment for the economy as a whole in 
the same way that it reduces unemployment for the 
"completers'!; (3) there is some evidence to suggest 
that the benefits of training are rather short- 
lived and consist principally of facilitating a 
rapid return to gainful employment for unemployed 
workers; and (4) not all of the benefits of the 
training programs are the result of the training 
per se --benefits may derive from a multitude of 
causes. 


Somers, Gerald G. 
1965 a "Training the Unemployed.'' Pp. 227 - 250 
in Joseph M. Beck (ed.), In Aid of the Unemployed. 
Baltimore: The Johns Hopkins Press. 


A general description of ARA and MDTA programs in the 
United States and an evaluation of government 
retraining based on surveys conducted in West Virginia. 


Because of the limited period of experience with 
retraining, the state of the economy and, the scale 

of the programs relative to the number of unemployed, 
government-subsidized retraining has thus far achieved 
only minor reduction in unemployment. 
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Various private training programs and the foreign 
training experience are also discussed. 


1965 b Retraining: an evaluation of gains and costs. Poxi271, - 
298 in Arthur M. Ross (ed.), Employment Policy and the 
Labor Market. Berkeley: University of California Press. 


A research evaluation of retraining in West Virginia 
yields findings which are favorable to the government- 
sponsored retraining of the unemployed. That is, 

the economic gains of the trainees resulting from 

the retraining appears to exceed the economic costs. 
When other nonquantifiable considerations are added 
(such as the social-psychological benefits accruing 

to the trainee who returns to the active labor 

market after a prolonged period of unemployment) , 

the results are even more impressive. 


1965 c Research Methodology in the Evaluation of Retraining 
Programs, Labor and Automation Bulletin No. 1, 
(IRRI Reprint Series No. 61). Madison: The 
University of Wisconsin. 


The author provides a comprehensive nine-point 
checklist of problem areas common to the eval- 

uation of retraining programs, a description of 

a Ford Foundation retraining project in West Virginia, 
and a number of suggestions of particular relevance to 
further research efforts. 


1966 a "An evaluation of government retraining programs 
for unemployed older workers in the U.S.A.'' Pp. 221 - 
223 in the Proceedings of the Seventh International 
Congress of Gerontology. Vienna, Austria, June 27 - 
Wully? 25219662 


In spite of the relatively greater need for retraining 
among unemployed older workers, current retraining 
efforts are generally inadequate - older workers being 
seriously under-represented in programs of the Area 
Redevelopment Act and the Manpower Development and 
Training Act. 


1966 b "Retraining the unemployed older worker.'' Pp. 109 - 
125 in Jaunita M. Kreps (ed.), Technology, Manpower 
and Retirement Policy. New York: The World 
Publishing Company. 
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Argues that government-sponsored retraining has been 
assigned a relatively limited role in approaching the 
employment and unemployment problems of the older 
worker. The net result is that older workers are 
grossly underrepresented in government retraining 
programs. 


1967 a "Evaluation of manpower development programs.'! 
Pp. 143 - 152 in Edward B. Jakubauskas and C. Phillip 
Baumel (eds.), Human Resources Development. Ames, lowa: 
lowa State University Press. 


This article evaluates two federal manpower development 
programs - the retraining program and the labor mob- 
i lity program. 


1967 b ''Our experience with retraining and relocation.'! 
Pp. 215 - 236 in Robert Aaron Gordon (ed.), Toward 
a Manpower Policy. New York: John Wiley & Sons, Inc. 


The author reviews the new government retraining 
programs and points out that the few detailed 
evaluations made of these indicate that they are 
effective in improving the economic status of the 
trainees. More sophisticated cost-benefit analyses 
will be required to determine whether the new programs 
are a significant factor in reducing unemployment and 
poverty. Also considered are the methodological 
shortcomings of the evaluations of these programs. 


Somers Gerald G. and Graeme H. McKechnie. 


1968 "Vocational retraining programs for the unemployed."' 
Pp. 25 - 35 in Gerald G. Somers (ed.), The Development 
and Use of Manpower. Washington: Industrial Relations 


Research Association. 


A collection of case studies endeavoring to evaluate 
some of the earliest attempts at retraining programs 
for the unemployed, established under federal, state, 
municipal and union-management auspices. The most 
significant conclusion deriving from the comparative 
analysis of these studies is that the retraining of 
unemployed workers is a sound social investment. 
Besides a strict dollars-and-cents appraisal of the 
gains of retraining, the studies make an attempt to 
incorporate such nonquantifiable benefits as the 
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quality of the training experience; the satisfactions 
accruing to both the trainees and their employers from 
the trainees! increased productivity; the changes in 
morale, motivation, work attitudes, social status and 
self-esteem resulting from retraining. 


Somers Gerald G. and Graeme H. McKechnie. 
1968 ‘Vocational retraining programs for the unemployed.'' 
Pp. 25 - 35 in Gerald G. Somers (ed.), The Development 


and Use of Manpower. Washington: Industrial Relations 


Research Association. 


A review of what we Know and what we do not know in the 
evaluation of retraining programs for the unemployed. 
The authors contend that traditional vocational 
education is not reaching the disadvantaged and many 
efforts to evaluate the benefits of vocational 
retraining are unreliable due to methodological 
deficiencies. 


Somers, Gerald G. and Ernst W. Stromsdorfer. 


1965 "A benefit-cost analysis of manpower retraining.'' 
Pp. 172 - 185 in the Proceedings of the Seventeenth 
Annual Meeting, Industrial Relations Research Assoc- 
jation. Madison: Wisconsin. 


An evaluation of the state retraining programs in West 
Virginia yields a favorable benefit-cost relationship. 
The West Virginia surveys (conducted in 1962) indicated 
that trainees enjoyed notable advantages in employment 
and earnings relative to ''comparable'' nontrainees. This 


was especially true for males where the average cost 
per trainee was quickly repaid by increased earnings. 


Somers, G.G. and W.D. Wood (eds.). 


1969 


Cost-Benefit Analysis of Manpower Policies, the Proceedings 
of a North American Conference. Kingston: Industrial 
Relations Centre, Queen's University. 


The proceedings of a conference concerned with the 
utilization of cost-benefit calculations in the 
evaluation of manpower policies and programs. The 
papers and discussions in this volume serve two 
important functions in the cost-benefit and manpower 


fields: 


(1) they provide a theoretical appraisal of a 


potentially powerful tool of economic analysis - one 
which is being increasingly used to evaluate a wide 
variety of public expenditures; and (2) they 
demonstrate the strengths and weaknesses of this 
economic tool in evaluating some of the most important 
manpower programs of the past decade. 
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Spaeth, Joe L. 
"A nationwide evaluation of MDTA institutional job 
training programs.'' National Opinion Research Center, 
Report No. 118. 


A nationwide research project conducted by the National 
Opinion Research Center attempts to determine the 

impact of MDTA institutional training on the economic 
situation of trainees. To facilitate this task, a 
comparison is made between MDTA trainees and a group of 
nontrainees who were unemployed about the same time the 
training programs began. 


Spargo, R.A. 
1961 "'Benefit-céost analysis and project evaluation.'' Pp. 299 - 
310 in Resources for Tomorrow, Volume |, Conference 
Background Papers, Montreal, October, 1961. 


A general discussion of benefit-cost analysis as a 
technique for assessing the economic effects of 
resource development projects undertaken by public 
agencies. The paper considers the basic concepts of 
the analysis, its methodology and its limitations. The 
principal limitations include: (a) the difficulty of 
translating program components into monetary terms; 
(b) the failure to take into consideration unemployed 
resources and the relationship between interest rates 
and price levels; and (3) the need for a clear and 
concise statement on the principles and procedures of 
measurement. 


Stevenson, Diantha. 
1967 Follow-Up Study of MDTA E & D Projects Conducted by 
Morgan State College. Washington, D.C.: Bureau 
of Social Science Research, Inc., May, 1967. 


A follow-up report on two Morgan State College projects 
- Morgan | devoted to vocational training and guidance 
and Morgan || emphasizing on-the-job training - 
designed to improve the occupational position of the 
underemployed and underprivileged. 


Striner, Herbert E. 
1966 "Research strategy for manpower policy.'' Pp. 229 - 240 
in Sar Levitan and Irving Siegal (eds.), Dimensions of 
Manpower Policy: Programs and Research. Baltimore: 
Johns Hopkins Press. 


Examines some of the broader research implications of 
the order-of-magnitude changes already registered or 
expected in the federal manpower commitment. Discusses 
five objectives or research strategy that will 
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prove important in the next half decade: (1) 
redefinition of manpower problems in a ''systems'! 


context; (2) clarification of new requirements 
for local socio-economic data; (3) increasing 
collaboration among the disciplines -- social, 


behavioral and physical; (4) increasing service of 
private, independent research organizations ina 
catalytic role; (5) development of new techniques to 
facilitate implementation of research results. 


Stromsdorfer, Ernst W. 
1968 "Determinants of economic success in retraining the 
unemployed: the West Virginia experience."' 
Journal of Human Resources 2, (Spring): 139 - 158. 


An evaluation of the post-training labor market 
experience of workers from a government sponsored 
retraining program in West Virginia from 1959 to 1964. 
Survey interviews of five groups (dropouts, did not 
reports, rejects, nontrainees and trainees) began in 
1962 and followups were done two years later. The 
author feels that retraining has a positive net effect 
on labor market success both in employment and 
earnings terms. 


Suchman, Edward A. 


1967 Evaluative Research. New York: Rusell Sage Foundation. 


An exhaustive and excellent discussion of the growth 
and current status of evaluation; the concepts, 
principles, types, and categories of evaluation; 

the conduct and design of evaluative research; the 
measurement of effects, the problems of program 
administration of evaluation studies. 


Therkildsen, Paul and Philip Reno. 
1968 "'Cost-benefit evaluation of the Bernalillo County 
work experience project.'' Welfare in Review 6 


(March =fADriq) 371 Sel2: 
A cost-benefit study of the Title V Work Experience 


and Training project, measuring inputs and outputs 
in a familial context. 
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1966 'An evaluation of the Upjohn Institute Study by Becker, 
Haber and Levitan: programs to aid the unemployed in 
the 1960's.'' From the Disadvantaged Poor: Education 


and Employment. United States: Chamber of Commerce 
Task Force on Economic Growth and Opportunity. 
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Twenty-six proposals to aid the unemployed are 
condensed into seven broad types of programs with 


three major divisions: (1) alleviative programs, e.g. 


unemployment insurance and programs of private 
industry; (2) curative programs, e.g. area 
redevelopment, public works and training; (3) 
programs which are both alleviative and curative, e.g. 
welfare programs and public employment services. 


U.S. Department of Labor, Manpower Administration. 
1970 The Neighborhood Youth Corps: A Review of Research, 
Monograph No. 13 Washington, D.C.: U.S. Government 
Printing Office. 


This survey reviewed various research studies which 
assessed the effectiveness of Neighborhood Youth Corps 
(NYC) projects. While there are three types of 
programs (in school, summer and out-of-school) with 
differing objectives, methods, and research approaches ; 
certain elements were found to be common to most 
programs, and an analysis of these elements provided 
the basis for an overall assessment of NYC. 


University of Chicago. 
1969 Evaluation of Youth Manpower Demonstration of the 
Woodlawn Organization. Chicago: School of Social 
Sciences Administration, University of Chicago. 


A study of gang delinquency in Woodlawn and of how a 
youth manpower development project was instrumental 
in preventing it. 


Vivien, Neal E. 
1961 "Evaluative criteria for distributive education: methods 
of evaluation.'' Business Education Forum 15 (April). 


A brief statement on the methods of evaluation of 
distributive education programs and students 
participating in these programs. Testing, use of 
student records, and on-the-job training evalu- 
ation are discussed. 


Walther, Regis H. and Margaret L. Magnusson. 
1967 A Retrospective Study of the Effectiveness of Out-of- 
School Neighborhood Youth Corps Programs in Four Urban 


Sites. Washington D.C.: Social Research Group, George 
Washington University. 
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A study of the effectiveness of the Out-of School, 
Neighborhood Youth Corps Programs in four urban areas 
reveals that the programs have enrolled disadvantaged 
youths (especially males) and are improving the work 
and community adjustment of participating youths. 
Compared to young persons in the control group, the 
NYC youths had more supplemental and remedial 
education, and were more frequently employed and 
self-supporting. 


Walton, Barbara J. 


1969 Second Annual Report and Evaluation of the Talent Corps 
College for Human Services. New York: Talent Corps, 
April, 1969. 


A description and evaluation of the Talent Corps (an 
institute to train women from low-income areas to 
serve as paraprofessionals in New York City) during 
the 1967-68 program period. 


Weber, Arnold R. 
1968 "Experiments in retraining a comparative study.'! 
Pp. 257 - 296 in G.G. Somers (ed.), Retraining the 


Unemployed. Madison, Wisconsin: University of Wis- 
consin Press. 


A comparative study of the different approaches to 
retraining the unemployed, emphasizing: (a) the 
various "models" of retraining; (b) the results 

of these different projects in terms of the perform- 
ance of trainees in the program and their subsequent 
experience in the labor market; (c) a cross-sectional 
analysis as a basis for broader inferences concerning 
public and private efforts to retrain the unemployed. 


Weber, Robert E. 
The Problem of Youth Training. Washington, D.C.: 
Office of Juvenile Delinquency and Youth Development. 


An evaluation of the youth employment programs under 
the Office of Juvenile Delinquency. 


Weinberg, Joseph L. 
1966 Report: Evaluation Study of Youth Training and Employment 
Project, East Los Angeles. Washington, D.C.: United 
States Department of Labor, Office of Manpower, Auto- 
mation and Training. 
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An evaluation of the East Los Angeles Youth Training 
and Employment Project service program, designed to 
help out-of-school, jobless youth prepare for employ- 
ment. 


Weisbrod, Burton A. 
1961 'The valuation of human capital.'' Journal of Political 
Economy (October): 425 - 436. 


Attempts to indicate the importance of developing 
estimates of human-capital values, and to discuss some 


of the difficulties -- at the methodological and data 
levels -- of obtaining them. 
1966 "Conceptual issues in evaluating training programs.'' 


Monthly Labor Review 89 (October): 1091 - 1097. 


A general discussion of problems involving the costs 
and benefits of manpower training programs, with 
suggestions regarding methods of measurement. The 
principal objectives of training programs may be 
expressed briefly as increased income and decreased 
unemployment but this presents analytic difficulties: 
(1) if increased and decreased refer to before and 
after the training program, there still remains the 
problem of evaluating the program on the basis of the 
possible future of the trainees if there had bean no 
program at all; (2) if goals are stated with 
reference only to trainees, favorable and unfavorable 
effects on third parties tend to be ignored. 


1968 ''Income redistribution effects and benefit - cost analysis." 
Pp. 177 - 209 in Samuel B. Chase (ed.), Problems in Public 
Expenditure Analysis. Washington, D.C.: The Brookings 
Institute. 


The paper seeks to direct attention to the means by 

which economists can assist in improving the quality 

of decisions on public expenditures by confronting the 
issue of a project's distributional effects in addition 
to its allocative-efficiency effects. The author 
concentrates on formulating the proper role of 
distribution considerations in a benefit-cost framework. 
Section one is devoted to developing the conceptual 

case for integrating income-distributional effects and 
allocative-efficiency effects in the evaluation of 

public expenditure projects, while sections-two and three 
consider some methods by which this end might be achieved. 
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methodological issues. Pp. 3 - 15 in G.G. Somers and W. 
D. Wood (eds), Cost-Benefit Analysis of Manpower Policies, 
The Proceedings of a North American Conference. Kingston: 
Industrial Relations Centre, Queen's University. 


Benefits from manpower programs may be judged in terms 

of the program objectives. While these objectives are 
stated in a wide variety of ways, they involve 

generally the achievement of (1) greater allocative 
efficiency, (2) enhanced economic stability (less infla- 
tion and less unemployment), and (3) improved distrib- 
utional equity. 


The author feels that manpower programs may, but are 
not likely to, produce benefits in excess of costs in 
either of the first two forms. Any evaluation of a 
manpower program should begin, therefore, with the 
presumption that the program is not economically 
efficient in the sense that benefits in the form of 
increased worker productivity (as measured by 
earnings) exceed the real cost of the program. 


Weissman, Murray. 
Research Report on the Effectiveness of Neighborhood 
Youth Corps. Cincinatti, Ohio: Ther Citizen's 
Committee on Youth. 


A national study of the effectiveness of the 

out-of-school Neighborhood Youth Corps Program to: 

(1) determine whether or not the experience in the NYC 

adds to the social, economic and educational improve- 

ment of its participants; (2) assess NYC's impact on 
participants! families and communities; and (3) identify 
NYC program components related to the changes in education, 
work and social adjustment. 


Weiss, Robert S.,and Martin Rein. 
Evaluation of Broad-Aim Programs. Boston: The Academy 
Committee of Arts and Science. 


This article considers the problems involved in the 
evaluation of broad-aim programs which are often 
characterized by unstandardized and large-scale 
interventions to the on-going systems. In particular, 
the experimental design in evaluation of broad-aim 
programs leads to severetechnical and administrative 
difficulties which include: (1) selecting satisfactory 
criteria; (2) an essentially uncontrolled situation; 
(3) understandardized treatments; (4) pointless use of 
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comparison groups; (5) experimental design which is 
limited in the information it can produce; (6) con- 
flicts over program modification; (7) researchers 
who are dependent on unreliable record keepers; 

(8) operationalizations which become leading goals; 
and (9) a research staff which is less well inform- 
ed than the action group. 


Weissman, Harold H. 

1969 An appraisal of youth employment programs.'' Pp. 110 - 
119 in Harold H. Weissman (ed.), Employment and Educ- 
ational Services in the Mobilization for Youth 
Experience. New York: Association Press. 


A general appraisal of the Mobilization for Youth 
Experience programs. Low income youths do need 

help in finding employment; however, the problem cannot be 
solved apart from the general problem of employment in 

the economy. The author feels that the soundest econ- 

omic policy is to keep youth in school. 
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A relatively detailed assessment of the first retraining 
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acceptance of training; (6) evaluation of the courses; 
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The author discusses problems facing a researcher 
attempting to develop an evaluation strategy for 

a social action agency. These include inferior 
methodological tools, severe field problems in 
implementing evaluations and problems in implement- 
ing new program ideas derived from evaluations and 
other sources, problems of integrating outcome 
evaluation results into the agency decision-making 
process and a basic weakness of R & D programs in 
producing outcome data. More competent evaluation 
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